{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. 为什么要引入核函数？（10分）\n",
    "2. 给出合页损失的数学形式并画出图形。（10分）\n",
    "3. 什么是支持向量？为什么SVM中只有一小部分的训练样本是支持向量（稀疏的）？（10分）\n",
    "4. 决策树中特征分裂的准则有哪些？（10分）\n",
    "5. SVM模型并没有概率解释。为了使SVM模型能输出概率，我们应该设置哪个参数？（10分）\n",
    "6. 采用train_test_split，从将数据集中随机抽取10000条记录，用于下述作业中模型的训练（原始数据集太大，剩余数据抛弃）。（10分）\n",
    "7. 根据6中得到的训练数据，训练线性SVM，并对超参数（正则惩罚项、C）进行超参数调优，并分析不同C对应的训练误差和交叉验证得到的测试误差的变化趋势。（20分）\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. 为什么要引入核函数？（10分）\n",
    "将低维空间非线性问题映射到高维空间编程线性问题进行处理。许多在低维空间难以处理的非线性分类问题，转换到高维空间和容易得到最优分类超平面。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. 给出合页损失的数学形式并画出图形。（10分）\n",
    "![link1]\n",
    "\n",
    "[link1]:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/liu/anaconda3/lib/python3.7/site-packages/matplotlib/font_manager.py:1331: UserWarning: findfont: Font family ['sans-serif'] not found. Falling back to DejaVu Sans\n",
      "  (prop.get_family(), self.defaultFamily[fontext]))\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAecAAAEyCAYAAADA/hjIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XmcnuO9+PHPNzMhNLYS5UQiilLtsXUOWj2KJCS2UKRSHE5pqofaal9iq732rY19K1KCIJbYWlVSidpVm9NN0JPYqVJJr98f1/TXaTrMJPPMXPfzzOf9es0rzzNzm+f7eko+ve77fu47UkpIkqTq6FN6AEmS9M+MsyRJFWOcJUmqGOMsSVLFGGdJkirGOEuSVDHGWZKkijHOkiRVjHGWJKlimku98DLLLJOGDBlS6uUlSepx06dPfzWlNKCj7YrFeciQIUybNq3Uy0uS1OMi4g+d2c7d2pIkVYxxliSpYoyzJEkVY5wlSaoY4yxJUsUYZ0mSKsY4S5JUMR3GOSL6RcQvIuLJiHg2Io5rZ5uFI+KGiJgREVMjYkh3DCtJUm/QmZXzB8CmKaW1gLWBERGxwTzb7AG8kVJaBTgLOLW2Y0qS1Ht0GOeUvdv6tG/rV5pns1HAla2PbwSGRkTUbMoOfPABnHYafPhhT72iJEndp1PHnCOiKSKeAGYBU1JKU+fZZCDwIkBKaQ7wFrB0O79nbERMi4hps2fP7trkbdx9Nxx6KIwZY6AlSfWvU3FOKc1NKa0NrACsFxGfn2eT9lbJ866uSSmNTym1pJRaBgzo8LrfnbbNNnDmmXDTTbDLLjBnTs1+tSRJPW6+bnyRUnozIh4ERgDPtPnRTGAQMDMimoElgNdrNWRnHHAAzJ0LBx8MTU1w1VXQXOy2HpIkLbgO8xURA4APW8O8CDCMfz3haxKwG/AIsANwf0rpX1bO3e2gg3KgDzssB/qKK/KfkiTVk86sLZcHroyIJvJu8Akppdsj4nhgWkppEnApcHVEzCCvmHfqtok7cOihebf2UUflMF92GfTx09ySpDrSYZxTSk8B67Tz/XFtHr8P7Fjb0RbckUfmQB97bN61PX68gZYk1Y+GPSo7blwO9Pe+l1fQF11koCVJ9aFh4xwBxx+fj0GffHIO9AUX5O9LklRlDRtnyCE+8cS8gj799LyL+5xzDLQkqdoaOs6QQ3zqqTnQZ52VV9BnnmmgJUnV1fBxhhziM87Iu7jPPjsH+vTTDbQkqZp6RZwhh/jss/MK+owz8i7uk0820JKk6uk1cYYc4vPOyyvoU0/NgT7hBAMtSaqWXhVnyB+nuvDCHOgTT8yBPvbY0lNJkvQPvS7OkAP9wx/mXdzHHZePQR99dOmpJEnKemWcIQf6kkvyCnrcuLyCPvzw0lNJktSL4wx5xXz55TnQRxyRA33wwaWnkiT1dr06zpADfeWVOdCHHJIDfcABpaeSJPVmvT7OkIN8zTU50AcemIO9776lp5Ik9VbGuVVzM/zoRznQ++2XA7333qWnkiT1Rt6nqY2+feH662GbbWCfffIZ3ZIk9TTjPI+FFoIJE2DLLWGvvfIZ3ZIk9STj3I6FF4Ybb4QRI2Ds2HxGtyRJPcU4f4R+/WDiRBg2DPbYA66+uvREkqTewjh/jEUWgVtugU02gd13zyeMSZLU3YxzBxZdFG67DTbaCHbdFW64ofREkqRGZ5w74e+B3nBD2HnnfDxakqTuYpw7qX9/uOMOWH99GDMGbr659ESSpEZlnOfDYovBnXdCSwuMHg2TJpWeSJLUiIzzfFp8cbjrLlhnHdhhB7j99tITSZIajXFeAEssAffcA2uuCdtvn2MtSVKtGOcFtOSSOdBrrAHbbgtTppSeSJLUKIxzF3zyk3DvvbDaavl63PfdV3oiSVIjMM5dtPTSOdCrrAJbbw0PPlh6IklSvTPONTBgQF41r7RSvmHGQw+VnkiSVM+Mc40su2wO9ODBMHIkPPxw6YkkSfXKONfQcsvB/ffDwIH5jlaPPFJ6IklSPTLONbb88jnQn/pUDvQvflF6IklSvekwzhExKCIeiIjnI+LZiNivnW02joi3IuKJ1q9x3TNufRg4EB54IJ8sttlmMH166YkkSfWkMyvnOcB3U0qfBTYA9o6INdrZ7qGU0tqtX8fXdMo6NGhQDvRSS8Hw4fDLX5aeSJJULzqMc0rplZTS462P3wGeBwZ292CNYMUVc6AXWwyGDYMnnyw9kSSpHszXMeeIGAKsA0xt58dfjIgnI+LOiPhcDWZrCEOG5GPQiy4KQ4fC00+XnkiSVHWdjnNE9AduAvZPKb09z48fB1ZMKa0FnAfc8hG/Y2xETIuIabNnz17QmevOyivnFfTCC+dAP/dc6YkkSVXWqThHRF9ymK9NKU2c9+cppbdTSu+2Pp4M9I2IZdrZbnxKqSWl1DJgwIAujl5fVlklB7qpCTbdFH71q9ITSZKqqjNnawdwKfB8SunMj9hmudbtiIj1Wn/va7UctBF85jM50ACbbAIvvFB2HklSNXVm5bwhsCuwaZuPSm0REXtFxF6t2+wAPBMRTwLnAjullFI3zVzXVl89X0ls7ty8gp4xo/REkqSqiVINbWlpSdOmTSvy2lXw9NM5zv365ZtlrLxy6YkkSd0tIqanlFo62s4rhBXy7/+e72b13nt5F/fvfld6IklSVRjngtZaKwf6nXfyKvoPfyg9kSSpCoxzYeusA1OmwBtv5EC/+GLpiSRJpRnnCmhpgXvugVdfzbu4X3qp9ESSpJKMc0Wstx7cfTfMmpUD/corpSeSJJVinCtkgw3gzjvh5ZfzLu4//an0RJKkEoxzxWy4IUyeDH/8Y77U56xZpSeSJPU041xBG20Ed9yRP141dCj0osuQS5IwzpW18cZw2235CmLDhsFrXgxVknoN41xhQ4fCrbfma3APGwavv156IklSTzDOFbfZZnDLLfk2k5ttBm++WXoiSVJ3M851YMQImDgRnnoqB/qtt0pPJEnqTsa5Tmy5Jdx4I/zylznWb79deiJJUncxznVkm21gwgR47DEYOTJfk1uS1HiMc53Zbju4/nqYOjWvpt99t/REkqRaM851aIcd4Npr4eGHYaut4M9/Lj2RJKmWjHOd+trX4Oqr4aGH8u7u994rPZEkqVaMcx37+tfhiivggQdg223h/fdLTyRJqgXjXOd23RUuuwzuvTcfjzbQklT/jHMD2H13GD8e7rorH4/+4IPSE0mSusI4N4g994Qf/CDfMGP0aPjrX0tPJElaUMa5gXzrW3D++TBpEuy0E3z4YemJJEkLwjg3mL33hnPOgZtvzieMGWhJqj/NpQdQ7e27L8ydCwceCE1NcM010Oz/0pJUN/wru0EdcADMmQOHHJIDfdVV+U9JUvUZ5wZ28ME50EcckcN8+eUGWpLqgXFucIcfnndxH310DvOll0IfzzSQpEozzr3AUUflFfRxx+Vjzz/8oYGWpCozzr3EMcfkQJ94Yl5BX3QRRJSeSpLUHuPcS0TACSfkXdynnJIDff75BlqSqsg49yIRcNJJeQX9/e/nXdxnn22gJalqjHMvEwGnnZYDffbZeQV9xhkGWpKqpMM4R8Qg4CpgOeBvwPiU0jnzbBPAOcAWwHvA7imlx2s/rmohAs48M+/iPuusvII+9VQDLUlV0ZmV8xzguymlxyNiMWB6RExJKT3XZpuRwKqtX+sDF7X+qYqKyJf5nDMHTj89B/rEEw20JFVBh3FOKb0CvNL6+J2IeB4YCLSN8yjgqpRSAh6NiCUjYvnWf1YVFZFPCps7F04+OQf6+ONLTyVJmq9jzhExBFgHmDrPjwYCL7Z5PrP1e8a54vr0yR+rmjMnn83d3AzjxpWeSpJ6t07HOSL6AzcB+6eU3p73x+38I6md3zEWGAswePDg+RhT3alPH7j44ryCPuaYfJLYkUeWnkqSeq9OxTki+pLDfG1KaWI7m8wEBrV5vgLw8rwbpZTGA+MBWlpa/iXeKqdPn3xpz7lz8xXFmpvh0ENLTyVJvVNnztYO4FLg+ZTSmR+x2SRgn4i4nnwi2Fseb64/TU1wxRU50Icdlp8fdFDpqSSp9+nMynlDYFfg6Yh4ovV7RwCDAVJKPwAmkz9GNYP8Uar/rv2o6gl/v73k3Ln5rlbNzbD//qWnkqTepTNna/+M9o8pt90mAXvXaiiV1dwM11yTA33AATnY3/lO6akkqffw3kRqV9++cN11sO22sO++cOGFpSeSpN7DOOsj9e0LN9wAW28Ne+8N48eXnkiSegfjrI+10ELw4x/DFlvAt74Fl11WeiJJanzGWR1aeGG46SbYfHPYc0+48srSE0lSYzPO6pR+/eDmm2HoUPjv/84njEmSuodxVqctsgjceitsvDHstls+YUySVHvGWfNl0UXhttvgP/8TdtkFJkwoPZEkNR7jrPn2iU/A7bfDl74EX/96Ph4tSaod46wF0r8/TJ4M660HO+0Et9xSeiJJahzGWQtsscXgrrvgC1+A0aPz7m5JUtcZZ3XJ4ovD3XfD2mvDDjvk1bQkqWuMs7psiSVyoD//edhuu/xYkrTgjLNqYqmlYMoUWGMNGDUK7r239ESSVL+Ms2rmk5/Mgf7MZ/L1uO+/v/REklSfjLNqapll4L77YOWVYaut4Cc/KT2RJNUf46yaGzAgB3rIENhyS3joodITSVJ9Mc7qFp/6VN6tvcIK+Y5WP/956YkkqX4YZ3Wb5ZbLgV5+eRgxAqZOLT2RJNUH46xu9W//Bg88AMsuC5ttBo89VnoiSao+46xuN3BgDvTSS+dAT59eeiJJqjbjrB4xaFAO9BJLwPDh8MQTpSeSpOoyzuoxK66YA92/PwwbBk89VXoiSaom46wetdJKOdD9+sHQofDMM6UnkqTqMc7qcSuvnAPdty9suik891zpiSSpWoyzilh11RzopqYc6F/9qvREklQdxlnFrLZa/hx0SjnQv/516YkkqRqMs4r67GdzoD/8EDbZBGbMKD2RJJVnnFXc5z6Xr8X9wQc50L/9bemJJKks46xKWHPNfA/o997Lgf7970tPJEnlGGdVxtpr5/tBv/12DvQf/1h6IkkqwzirUtZdNwf6jTdyoGfOLD2RJPU846zKaWmBu++GV1/NgX755dITSVLPMs6qpPXXh7vugj/9KQf6lVdKTyRJPafDOEfEZRExKyLavdBiRGwcEW9FxBOtX+NqP6Z6oy9+Ee68E156KX8O+v/+r/REktQzOrNyvgIY0cE2D6WU1m79Or7rY0nZl78Mkyfnk8M23RRmzSo9kSR1vw7jnFL6KfB6D8witWujjeD22+F3v8t3s3r11dITSVL3qtUx5y9GxJMRcWdEfO6jNoqIsRExLSKmzZ49u0Yvrd5gk03gttvgN7/JgX7ttdITSVL3qUWcHwdWTCmtBZwH3PJRG6aUxqeUWlJKLQMGDKjBS6s3GToUbr013yRj+PD8cStJakRdjnNK6e2U0rutjycDfSNimS5PJrVjs81g4kR49tn8+M03S08kSbXX5ThHxHIREa2P12v9ne50VLfZYgu46SZ48knYfHN4663SE0lSbXXmo1TXAY8Aq0XEzIjYIyL2ioi9WjfZAXgmIp4EzgV2Siml7htZgq22gh//GB5/HEaOhHfeKT2RJNVOlOpoS0tLmjZtWpHXVuOYOBFGj4YNNsgXLenfv/REkvTRImJ6Sqmlo+28Qpjq2le/CtddB48+CltuCX/+c+mJJKnrjLPq3o47wjXXwM9+BltvnW87KUn1zDirIey0E1x1FTz4IIwaBX/5S+mJJGnBGWc1jJ13hssvh/vug223hfffLz2RJC0Y46yGsttucMklcM89+Xj0Bx+UnkiS5p9xVsP5xjdg/Ph8R6sddoC//rX0RJI0f4yzGtI3vwkXXZRvmDF6tIGWVF+MsxrWXnvBeefl63GPGQMfflh6IknqHOOshrbPPnD22fliJTvvDHPmlJ5IkjrWXHoAqbvttx/MnQvf/S40NcHVV0Oz/+ZLqjD/ilKvcOCBedV86KE50Fdemf+UpCoyzuo1DjkkB/rII3OYL7vMQEuqJuOsXuWII/Iu7nHj8q7tiy+GPp55IalijLN6naOPzivo44/PK+cf/MBAS6oW46xe6dhjc6BPOikH+sILIaL0VJKUGWf1ShHwve/lQJ92Wt7Ffe65BlpSNRhn9VoRcMop+Rj0GWfkFfRZZxloSeUZZ/VqEXD66XkFfc45eQV9+ukGWlJZxlm9XkReMc+Z848V9CmnGGhJ5RhniRzi887Lu7j/fgz6e98z0JLKMM5Sqwi44IIc6JNOyoE+7rjSU0nqjYyz1EafPvlzz20/Bz1uXOmpJPU2xlmaR58++cphc+fCMcfkFfQRR5SeSlJvYpyldvz92ttz5+ZrcTc352tzS1JPMM7SR2hqgiuuyIE+9NAc6AMPLD2VpN7AOEsfo7k53/+57f2g99uv9FSSGp1xljrQ3AzXXpsDvf/++fnee5eeSlIj8148Uif07QvXXQejRsE+++QzuiWpuxhnqZMWWggmTICttoJvfzuf0S1J3cE4S/NhoYXgxhth5EgYOzaf0S1JtWacpfm08MIwcSJsthnsuSdcdVXpiSQ1GuMsLYB+/eCWW2DTTWH33fMJY5JUKx3GOSIui4hZEfHMR/w8IuLciJgREU9FxLq1H1OqnkUWgUmT4Ctfgf/6L7j++tITSWoUnVk5XwGM+JifjwRWbf0aC1zU9bGk+rDoonD77fDlL8Muu8CPf1x6IkmNoMM4p5R+Crz+MZuMAq5K2aPAkhGxfK0GlKruE5+AO+6ADTaAMWPy8WhJ6opaHHMeCLzY5vnM1u/9i4gYGxHTImLa7Nmza/DSUjX07w933gnrrQdf+xrcemvpiSTVs1rEub3b0af2NkwpjU8ptaSUWgYMGFCDl5aqY7HFcqDXXRd23DHv7pakBVGLOM8EBrV5vgLwcg1+r1R3llgC7r4b1loLtt8+x1qS5lct4jwJ+K/Ws7Y3AN5KKb1Sg98r1aUll4R77oHPfQ622y4/lqT50ZmPUl0HPAKsFhEzI2KPiNgrIvZq3WQy8FtgBnAx8D/dNq1UJ5ZaCqZMgdVXz9fjvvfe0hNJqicd3pUqpTSmg58nwHv0SPNYeukc5U03hW22yWd0b7JJ6akk1QOvECZ1o2WWyYFeaaV8w4yf/rT0RJLqgXGWutmyy8L998PgwbDFFvDww6UnklR1xlnqAZ/6VA70wIEwYgQ88kjpiSRVmXGWesjyy+dAL7ccbL45TJ1aeiJJVWWcpR40cCA88AAMGJADPW1a6YkkVZFxlnrYCivkQC+1FAwfDo8/XnoiSVVjnKUCBg/OgV58cRg2DJ54ovREkqrEOEuFDBmSA92/fw7000+XnkhSVRhnqaBPfzqfJNavHwwdCs8+W3oiSVVgnKXCVlklB7q5OV9N7PnnS08kqTTjLFXAZz6TAx2RA/3CC6UnklSScZYqYvXVc6D/9rd8De7f/Kb0RJJKMc5ShayxBtx3H3z4YQ70//5v6YkklWCcpYr5/OdzoP/ylxzo3/2u9ESSeppxlipozTXz3azefTcH+g9/KD2RpJ5knKWKWmcdmDIF3norB/rFF0tPJKmnGGepwr7wBbjnHnjttRzol14qPZGknmCcpYr7j/+Au++GWbNyoF9+ufREkrqbcZbqwAYbwF13wSuv5M9B/+lPpSeS1J2Ms1QnvvQluPNOmDkzB3rWrNITSeouxlmqI1/+MtxxB/z+9znQs2eXnkhSdzDOUp35ylfg9tvzBUqGDoVXXy09kaRaM85SHdp0U5g0CX79axg+HF5/vfREkmrJOEt1avhwuPVWeO65/PiNN0pPJKlWjLNUxzbfHG6+GZ5+Oj9+883SE0mqBeMs1bkttoCbboInnoARI+Dtt0tPJKmrjLPUALbeGiZMgOnTYeRIeOed0hNJ6grjLDWIbbeF66+HqVPzavrdd0tPJGlBGWepgWy/PfzoR/Dzn8NWW8Gf/1x6IkkLwjhLDWb0aLjmGnjooby7+733Sk8kaX4ZZ6kBjRkDV14JDz4Io0bBX/5SeiJJ88M4Sw1ql13g8svhvvtgu+3g/fdLTySpszoV54gYEREvRMSMiDisnZ/vHhGzI+KJ1q89az+qpPm1225w8cX5lpPbbw8ffFB6Ikmd0dzRBhHRBFwADAdmAo9FxKSU0nPzbHpDSmmfbphRUhfssQfMnQvf+hbsuCPceCMstFDpqSR9nM6snNcDZqSUfptS+itwPTCqe8eSVEtjx8KFF8Jtt8HXvgYfflh6IkkfpzNxHgi82Ob5zNbvzWv7iHgqIm6MiEHt/aKIGBsR0yJi2mzvdSf1qG9/G849F265JZ8wZqCl6upMnKOd76V5nt8GDEkprQncC1zZ3i9KKY1PKbWklFoGDBgwf5NK6rLvfAfOPDNf7nOXXWDOnNITSWpPh8ecySvltivhFYCX226QUnqtzdOLgVO7Ppqk7nDAAfkY9MEHQ1MTXH11/lNSdXQmzo8Bq0bESsBLwE7A19tuEBHLp5ReaX26DfB8TaeUVFMHHZQDfdhh0NycP3JloKXq6DDOKaU5EbEPcDfQBFyWUno2Io4HpqWUJgH7RsQ2wBzgdWD3bpxZUg0cemjerX3UUTnMl14KfbzygVQJnVk5k1KaDEye53vj2jw+HDi8tqNJ6m5HHpkDfeyxOdDjxxtoqQo6FWdJjeuYY/Iu7hNOyIG+6CIDLZVmnCVx3HF5BX3yyfkY9PnnQ7T3OQ1JPcI4SyICTjwxB/r00/MK+pxzDLRUinGWBOQQn3pqDvRZZ+VAn3mmgZZKMM6S/r8IOOOMfAz67LPzLu7TTjPQUk8zzpL+SUQO85w58P3v50CfdJKBlnqScZb0LyLgvPPyCvqUU/Iu7hNOMNBSTzHOktrVp0++k9Xcuflksb5988euJHU/4yzpI/XpAz/84T9fqOSoo0pPJTU+4yzpY/XpA5dcklfQRx+dj0EfdljpqaTGZpwldaipKd8cY+5cOPzw/Pzgg0tPJTUu4yypU5qa4Morc6APOSSvoA84oPRUUmMyzpI6rbkZrrkmB/rAA/Pz73yn9FRS4zHOkuZLczP86Ec50Pvum1fU//M/paeSGov3npE03/r2heuvh222gb33zmd0S6od4yxpgSy0EEyYAFtuCXvtBZdeWnoiqXEYZ0kLbOGF4cYbYcQI+OY34YorSk8kNQbjLKlL+vWDiRNh2DD4xjfg6qtLTyTVP+MsqcsWWQRuuQU22QR23z2fMCZpwRlnSTWx6KJw222w0Uaw665www2lJ5Lql3GWVDN/D/SGG8LOO+fj0ZLmn3GWVFP9+8Mdd8AGG8CYMXDzzaUnkuqPcZZUc4stBpMnQ0sLjB4NkyaVnkiqL8ZZUrdYfHG46y5YZx3YYYe8mpbUOcZZUrdZYgm45x5Yc0346ldzrCV1zDhL6lZLLpkDvcYasO22MGVK6Ymk6jPOkrrdJz8J994Lq62Wr8d9//2lJ5KqzThL6hFLL50DvcoqsNVW8JOflJ5Iqi7jLKnHDBgA990HK60EW2wBDz1UeiKpmoyzpB617LI50IMHw8iR8PDDpSeSqsc4S+pxyy2XjzsPHJgD/eijpSeSqsU4Sypi+eVzoJddFjbfHH7xi9ITSdXRqThHxIiIeCEiZkTEYe38fOGIuKH151MjYkitB5XUeAYOhAcegGWWgc02g+nTS08kVUOHcY6IJuACYCSwBjAmItaYZ7M9gDdSSqsAZwGn1npQSY1p0KAc6KWWguHD4Ze/LD2RVF5zJ7ZZD5iRUvotQERcD4wCnmuzzSjg2NbHNwLnR0SklFINZ5XUoAYPzoH+yldg2DA477x8Aw2ptJEjoW/fnn/dzsR5IPBim+czgfU/apuU0pyIeAtYGni17UYRMRYYCzB48OAFHFlSIxoy5B+B3nnn0tNI2Ztv5svQ9rTOxDna+d68K+LObENKaTwwHqClpcVVtaR/8ulPw3PPwW9+U3oSKSu1B6czcZ4JDGrzfAXg5Y/YZmZENANLAK/XZEJJvcpii8G665aeQiqrM2drPwasGhErRcRCwE7AvHdnnQTs1vp4B+B+jzdLkrRgOlw5tx5D3ge4G2gCLkspPRsRxwPTUkqTgEuBqyNiBnnFvFN3Di1JUiPrzG5tUkqTgcnzfG9cm8fvAzvWdjRJknonrxAmSVLFGGdJkirGOEuSVDHGWZKkijHOkiRVjHGWJKlijLMkSRUTpS7kFRGzgT/U8Fcuwzw32tAC8X3sOt/DrvM97Drfw67rjvdwxZTSgI42KhbnWouIaSmlltJz1Dvfx67zPew638Ou8z3supLvobu1JUmqGOMsSVLFNFKcx5ceoEH4Pnad72HX+R52ne9h1xV7DxvmmLMkSY2ikVbOkiQ1BOMsSVLFNEScI2LniHiq9evnEbFW6ZnqTUSsHhGPRMQHEXFQ6XnqUUSMiIgXImJGRBxWep56FBGXRcSsiHim9Cz1KiIGRcQDEfF8RDwbEfuVnqneRES/iPhFRDzZ+h4e1+MzNMIx54j4EvB8SumNiBgJHJtSWr/0XPUkIpYFVgS2Bd5IKX2/8Eh1JSKagF8Dw4GZwGPAmJTSc0UHqzMRsRHwLnBVSunzpeepRxGxPLB8SunxiFgMmA5s67+LnRcRAXwipfRuRPQFfgbsl1J6tKdmaIiVc0rp5ymlN1qfPgqsUHKeepRSmpVSegz4sPQsdWo9YEZK6bcppb8C1wOjCs9Ud1JKPwVeLz1HPUspvZJSerz18TvA88DAslPVl5S92/q0b+tXj65kGyLO89gDuLP0EOp1BgIvtnk+E/9CVGERMQRYB5hadpL6ExFNEfEEMAuYklLq0fewoeIcEZuQ43xo6VnU60Q736v/Y0aqWxHRH7gJ2D+l9HbpeepNSmluSmlt8p7Y9SKiRw+z1G2cI2LviHii9evfImJN4BJgVErptdLz1YN538PS89S5mcCgNs9XAF4uNIt6udbjpDcB16aUJpaep56llN4EHgRG9OTr1m2cU0oXpJTWbv1/Ns3ARGDXlNKvC49WN9q+hyklQ9I1jwGrRsRKEbEQsBNWnuMkAAAAvklEQVQwqfBM6oVaT2a6lHyS7Jml56lHETEgIpZsfbwIMAz4VY/O0CBna18CbM8/bkE5x7uxzJ+IWA6YBiwO/I18xuwa7g7rvIjYAjgbaAIuSymdWHikuhMR1wEbk2/V93/AMSmlS4sOVWci4svAQ8DT5P+WAY5IKU0uN1V9ad0TeyX5v+U+wISU0vE9OkMjxFmSpEZSt7u1JUlqVMZZkqSKMc6SJFWMcZYkqWKMsyRJFWOcJUmqGOMsSVLF/D+zZ4y7bAiOkQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import math\n",
    "import matplotlib.pyplot as plt\n",
    "plt.figure(figsize=(8, 5))\n",
    "x = np.linspace(start=-2, stop=3,num =1001,dtype=np.float)\n",
    "y_hinge = 1.0 - x\n",
    "y_hinge[y_hinge < 0] = 0\n",
    "plt.plot(x, y_hinge, 'b-',mec='k', label='Hinge Loss')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. 什么是支持向量？为什么SVM中只有一小部分的训练样本是支持向量（稀疏的）？（10分）  \n",
    "支持向量是利用一个超平面把特征空间分隔成两个区域，超平面的方向由法向量w确定,它的位置由阈值b确定。找到最大化间隔的超平面将样本分类正确。  \n",
    "因为只有一小部分的样本在最大化间隔的超平面之内。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. 决策树中特征分裂的准则有哪些？（10分）\n",
    "ID3:信息增益最大(对标签Y提供信息最多的特征),倾向于选择取值多的特征进行分裂  \n",
    "C4.5:ID3的改进,信息增益率最大  \n",
    "CART: 分类:GINI指数最小,回归:均方误差最小"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. SVM模型并没有概率解释。为了使SVM模型能输出概率，我们应该设置哪个参数？（10分）\n",
    "probability=True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. 采用train_test_split，从将数据集中随机抽取10000条记录，用于下述作业中模型的训练（原始数据集太大，剩余数据抛弃）。（10分）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10000, 95)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/liu/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_split.py:2026: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from matplotlib import pyplot as plt\n",
    "train=pd.read_csv(\"Otto_FE_train_tfidf.csv\")\n",
    "#按照题目采用train_test_split抽取数据\n",
    "from sklearn.model_selection import train_test_split\n",
    "train,train_drop=train_test_split(train,train_size=10000,random_state=0)\n",
    "#train=train.sample(n=10000)\n",
    "print(train.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7. 根据6中得到的训练数据，训练线性SVM，并对超参数（正则惩罚项、C）进行超参数调优，并分析不同C对应的训练误差和交叉验证得到的测试误差的变化趋势。（20分）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/liu/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_split.py:2026: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    " #输入输出分开\n",
    "y_train=train['target']\n",
    "X_train=train.drop([\"id\",\"target\"],axis=1)\n",
    "#保留特征名字\n",
    "feat_names=X_train.columns\n",
    "#数据用稀疏矩阵表示，模型训练变快\n",
    "from scipy.sparse import csr_matrix\n",
    "X_train=csr_matrix(X_train)\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train_part,X_val,y_train_part,y_val=train_test_split(X_train,y_train,train_size=0.8,random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [],
   "source": [
    "#超参数调优，penalty，C\n",
    "from sklearn.svm import LinearSVC\n",
    "def fit_grid_point_Linear(penalty,C,X_train,y_train,X_val,y_val):\n",
    "    SVC1=LinearSVC(penalty=penalty,C=C, dual=False)#l1正则dual需要设置为False\n",
    "    SVC1=SVC1.fit(X_train_part,y_train_part)\n",
    "    accuracy_train=SVC1.score(X_train,y_train)\n",
    "    accuracy=SVC1.score(X_val,y_val)\n",
    "    print(\"penalty={} and C={} :,accuracy_train={}, accuracy_val={}\" .format(penalty,C,accuracy_train,accuracy))\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "penalty=l1 and C=0.1 :,accuracy_train=0.74375, accuracy_val=0.732\n",
      "penalty=l2 and C=0.1 :,accuracy_train=0.756125, accuracy_val=0.743\n",
      "penalty=l1 and C=1.0 :,accuracy_train=0.769, accuracy_val=0.7525\n",
      "penalty=l2 and C=1.0 :,accuracy_train=0.769875, accuracy_val=0.754\n",
      "penalty=l1 and C=10.0 :,accuracy_train=0.772625, accuracy_val=0.753\n",
      "penalty=l2 and C=10.0 :,accuracy_train=0.7725, accuracy_val=0.753\n",
      "penalty=l1 and C=100.0 :,accuracy_train=0.773, accuracy_val=0.751\n",
      "penalty=l2 and C=100.0 :,accuracy_train=0.772875, accuracy_val=0.753\n",
      "penalty=l1 and C=1000.0 :,accuracy_train=0.772875, accuracy_val=0.7515\n",
      "penalty=l2 and C=1000.0 :,accuracy_train=0.772875, accuracy_val=0.7515\n",
      "penalty=l1 and C=10000.0 :,accuracy_train=0.77275, accuracy_val=0.7515\n",
      "penalty=l2 and C=10000.0 :,accuracy_train=0.772875, accuracy_val=0.751\n"
     ]
    }
   ],
   "source": [
    "#需要调整的参数\n",
    "penalty_s=['l1','l2']\n",
    "C_s=np.logspace(-1,4,6)#吧10的-1次方到10的4次方分成6份\n",
    "accuracy_s=np.matrix(np.zeros(shape=(6,2)),float)\n",
    "for i, oneC in enumerate(C_s):\n",
    "    for j,onepenalty in enumerate(penalty_s):\n",
    "        tmp=fit_grid_point_Linear(onepenalty,oneC,X_train_part,y_train_part,X_val,y_val)\n",
    "        accuracy_s[i,j]=tmp\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'accuracy')"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/liu/anaconda3/lib/python3.7/site-packages/matplotlib/font_manager.py:1331: UserWarning: findfont: Font family ['sans-serif'] not found. Falling back to DejaVu Sans\n",
      "  (prop.get_family(), self.defaultFamily[fontext]))\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VNd99/HPT/sCWtDGIhaxg8HGBhO8L3gB42zPkzYmtpO0eR43bpw2cerEaRsnTdOnztK4beKkcZvEedLGjpvkaVwjjLEBrzgGO9gCxI5tBCMkJAQIof08f9wrGGRJDNLcmZHm+3695qW5954591yW+en87jnnmnMOERGRwUqJdwNERGR4UyAREZEhUSAREZEhUSAREZEhUSAREZEhUSAREZEhUSAREZEhCTSQmNkyM9tpZnvM7P4+jj9kZlv81y4zawo71hV27Mmw/Y+a2f6wYwuCvAYRERmYBTUh0cxSgV3AjUANsAlY6Zzb3k/5zwIXO+f+2N9uds6N6qPco8BTzrlfBdJwERE5L2kB1r0Y2OOc2wdgZo8DHwT6DCTASuCrQTSkuLjYTZkyJYiqRURGrNdff/2Ic67kXOWCDCQTgANh2zXA+/oqaGaTgQpgXdjuLDPbDHQCDzrn/ivs2N+Z2QPAc8D9zrm2Puq8C7gLYNKkSWzevHko1yIiknTM7J1IygV5j8T62NdfHu024FfOua6wfZOcc4uAjwH/aGbT/P1fBmYDlwJjgC/1VaFz7hHn3CLn3KKSknMGVBERGaQgA0kNMDFsuxw41E/Z24DHwnc45w75P/cBG4CL/e2Q87QBP8VLoYmISJwEGUg2ATPMrMLMMvCCxZO9C5nZLKAQ2Bi2r9DMMv33xcAV+PdWzGyc/9OADwFbA7wGERE5h8DukTjnOs3sHmANkAr8xDm3zcy+Dmx2zvUElZXA4+7s4WNzgB+ZWTdesHswbLTXf5hZCV7qbAvw6cG0r6Ojg5qaGlpbWwfz8ZjLysqivLyc9PT0eDdFROQsgQ3/TSSLFi1yvW+279+/n9GjR1NUVITXuUlczjkaGho4ceIEFRUV8W6OiCQJM3vdv1c9oKSd2d7a2josggiAmVFUVDRsek8iklySNpAAwyKI9BhObRWR5BLkPBIZjnY9A417oWg6FE2D/EmQqn8mItI/fUPE0ahRo2hubgZg2bJlvPrqq1x55ZU89dRT8WlQ/S54/GPQ3XFmX0o6jKmAMdO8wFI0/cxr9FhQT0kk6SmQJIj77ruPlpYWfvSjH8WnAc7BqnshPQc+tQZONUHDHu/VuBca9sLeddAVtohAei4UTT07uPQEnJwx8bkOEYk5BZIEsXTpUjZs2BC/BlT9J7z9Iqz4LpTO8fZNvuzsMt3dcLzGDzB+cGnYA4e2wPbfgus+UzZ7TFiACQs2Y6ZCRm7srktEAqdAAvzNf29j+6HjUa1z7vg8vvr+C6JaZ2BOHYU1fwkTFsLCP+q/XEoKFEzyXtOuP/tYZzs0vXOmF9MTbPZtgDd/cXbZvAleQAnvyRRNh8LJkKp5MiLDjQKJwHNfh5YGuOPXXrAYjLQMKJ7hvXpra4bGfWeCS6Pfk9n2/6C16Uw5S/WCyengMs1PlU33gs9g2yYigVIggeHTcwhCzWbY/FNYcjeMuyiYc2SOgnEXeq/eWhrDUmVhPZm3X4KOljPl0rLCbvj3uumfU6Sb/iJxpECSzLo64anPeaOvrvvL+LQhZwzkLIaJvdbedA5OhM4OLg17oa4adlZCd+eZsln5Z3ouPT2Znp+Zo2N7PSJJSIEkQVx11VXs2LGD5uZmysvL+fGPf8zNN98c7ElfewRqq+APfpZ4X7hmkDfee1Vcffaxrk7/fkxYmqxhD7y7EaqeOLvsqLKzg0tPwBlTAWmZsbsekRFMgSSOeuaQALz44ouxPfnxQ7D+72D6jTD3g7E991Clpp1JcfXWcQoa9/fqyezB7VyNnaw/XcxZCi3Z4zmaPYn6jHJyRxcwaUwuWenJch/GoHgmTLwUCiuUGpQhUSBJVk/f76WHbvn2sPsSae3ooqmlg6Mt7TS1dNDU0s7R09ve+6aWcRxtKeZoyyUca+mg6VQHud3NTLFaKixERUotFZ21VDSHmGa/J5t2ADrNW44mxf8jGV5/MufBdZ8Zrp1bAuWXQvkiKF8MEy7REG05LwokyWj3Wm/ex/V/7aV44qS723G8tePsIHDSe3/slPfzaE+gOHkmYJzq6Oq3zqz0FAqyMyjISacwJ4PZY0dTkJNBob/d874gJ91/n0FWZhpvHTzG+h11PLejjuqQNxS8vDCbpbNLuW52KUumFpGVnhqrP5rgdXVCfTUceM0bcFHzmnfvCbzRc2VzvaAycbEXZMZMHXa/cEjsJO0y8tXV1cyZMydOLRqcqLS54xQ8/D7v/sCnX4rafYJT7V0D9hB6jh1taedYy5lg0d3PP78Ug/zsni//M0HAe38mCJx+n+uVicaX/aGmU6zfWce66jpe3nuE1o5ustNTuWJ6MUvnlHLdrFLG5mcN+TwJp6XxTFA58BocfB3a/fRrTlGvXstCbzSejGiRLiOvHkmyeeE73o3qT/x3n0Gku9vRdOq9PYSmlg6aTp3dQwgPDm2d3X2czJOdnnrWF/74guz39BAKczLI938W5qSTl5VOSkp8fgMeX5DN7e+bzO3vm0xrRxcb9zawbkcd63bU8Wz1YQAuGJ93urdyUXlB3NoaVTljYOZN3guguwvqd5zda9n1tHfMUqB0rhdcenotRdPVa0lS6pEMI0Nuc/1O+OEVMO9/wv/oe02v2x7ZyKv7Gvs8lppiFGSnn1cPIT87fcSkhJxz7Dx8wgsq1XW88e5Ruh0Uj8rgmpmlLJ1TylUzihmdNYJn57c0wsE3zu61tPmrQmQX+r0W/zVhIWTlxbe9MiTqkcjZnINVX4CMHLjpG30WeafhJK/ua+RDC8ZzzayS9wSH0ZlpI+M370EyM2aPzWP22Dz+9NrpHD3ZzvO76lm3o46122v59Rs1pKUYiyvGcP3sUq6fXcrUkhGW/skZAzNu8F7grb92ZKffa9nkvXY/4xc2b922072WxV6vRSsUjDgKJHHUs4z8li1buPvuuzl+/Dipqan81V/9FR/96Eeje7K3fuktynjrQzCqpM8ilVW1AHzhpllMHJMT3fOPQIW5GXzo4gl86OIJdHZ188a7TTy34zDrquv4xqpqvrGqmori3NNB5dIpY8hIG2FfoikpXrAonQMLP+HtO9Xk9VRqNnkBZtt/wRs/845lFZy5z1K+yHtl5cev/RIVSm3FUU8g2bVrF2bGjBkzOHToEAsXLqS6upqCgoKzyg+6zS2N8P1LvRFaf/xMv78Rvv97L5Fi8Nt7rhzM5UiYA40tp++rbNzXQHtnN6My07hqRjHXzy7l2lmllIxOkgmR3d3QsPvsXktdNeAAg5JZZ/daimeq15IglNoaRmbOnHn6/fjx4yktLaW+vv49gWTQnvu6t8Lvrf/V73/QdxtaqDp4jC8vnx2dcya5iWNy+MTlU/jE5VNoae/k5T0NrNtxmHU76li9tRYzuLC8gKV+b+WC8Xkj93HKKSlesCiZBZfc6e1rPebfa/F7LdX/Db//uXcsMx/KF/q9lku999mF8Wu/nJMCCcDq+72lQqJp7HxY/uB5f+y1116jvb2dadP6mLU9GAdeg9d/Cks+47WpH6u3hgC4Zf646JxXTsvJSOPGuWXcOLcM5xzbDh0/3Vt56NldfHftLsryMrlulhdUrpxRTE7GCP+vmZUP067zXuD1Whr3nt1reeFbZyZNFvf0Wi71AkzJLEgZGYM4RoIR/q91eAmFQtx555387Gc/IyUaXfuuTnjqXhg9Hq778oBFK6tCzJ+Qr3sjATMz5k3IZ96EfP5s6QyONLexYWc963Yc5qm3Qjy+6QAZaSksmVp0ureSFH8nKSlnHkNw8e3evrYTYfdaNnkTJrf8u3csY3SvXssiPZUzjhRIYFA9h2g7fvw4K1as4Bvf+AZLliyJTqWv/QgOV8Ef/nzARRlrjrbwZs0xvrhsVnTOKxErHpXJRxaW85GF5bR3drP57Uae21HH+h11fPXJbXz1yW3MKB3F9XNKuX5WKQsnF5KWmiT3DzJHw9RrvRd4Iw8b9/m9Fr/n8uJ3zvRaimac3WspnaNeS4wokCSA9vZ2PvzhD/Pxj3+cP/iDP4hOpcdqYN3fwYybYM77Byy62h+ttUJprbjKSEvh8unFXD69mK/cOpf9R076KbDD/OSl/fzo+X3kZaVxzaxSls4u5ZqZJRTmZsS72bFjdmaxzgUrvX1tzXDojTO9lt1rzjyRM2OUt25Y+WJc+SLayi6hO3tM0k2azEpLDXzYvgJJAnjiiSd44YUXaGho4NFHHwXg0UcfZcGCBYOv9On7wXVFtCjjqqoQF4zPY3KRFupLJBXFuXzqygo+dWUFJ1o7eGn3EZ7bUceGnXX895uHSDG4ZFKh11uZXcqsstEj94Z9mM6ubppOhS/FM5OmrAqOTvgQRwvaST32NkVH32RCcxVT36mmYv93SaObLKDdpXKMURx1ozjKaJqc976J0f6+URxzvY4zis5h/FX57L3XML002PlMw/dPZwToWUb+jjvu4I477ohexbvWeKNglj4AhVMGLHqw6RRbDjRx381KayWy0VnpLJ8/juXzx9Hd7Xjr4LHTvZVvPb2Tbz29kwkF2Vw3u4Sls8u4bFriLzLpnKO5rdNfn61nkc72s1Z2PmvhTn/fidbOfutMTzVvxYXsyyjMu4aCsemUZnUx2+1lSvtucjuOktXRRGbnMSZ2HGN6RxNZHW+T1XGMVNfRb73tqbm0pufTlp5Pa1o+rekFtKXnedvpBbSm5Z9575drTx2VEL2fohj0WhVIRpr2Fqj8C2+Uy2WfPWfx1VXeaK3l88YG3TKJkpQUY8HEAhZMLODeG2dy+Hgr6/1RYL954yD//uq7ZKWncMW04tO9lXH52YG2qb2zm6ZT/pf/yfAvf2+NtqaT7w0Ox06109HV/zy20Vlpp1dWKMzJYGpx7unVFgpyzizVc3phz9wMcjNS++mVvW/gC3AO2k/CqUZvqHxLo/e+xdvOaGkk4/R2I7TshOON3jDm/qSkecOWs8d4P3PGeO9z/H2nt3v9TBt+6UoFkpHmhW9D07vwyVUR/YNcvbWW2WNHj7ylPJJIWV4Wty2exG2LJ9Ha0cXv9jf6S+If5rkddQDMGZfH9bNLuH52GQsmFpDaT87cOcfx1k6aBuwZnN1DaGrpoLmt/15CRlpK2CKd6UwrGUVhbnrYkv691mnLSSc/Oz22gwrMvNWMM0dBwaTIP9fd5c3kPyvI9PXzqPf/8tAWb19na/91puf6QaWw/2DTc7ynTGZ+XCdxJnUgcc4Nm5xyRCsQ1O2AV74HF30Mppx7dnro2Clef+coX7hx5jnLyvCQlZ7KNTNLuGZmCV99/1z21jfzXLX3nJV/eX4fD6/fy5jcDK6aUUxGasp7g8KpDrr6WdvfDPKyzizSWTIqk5mlZ573UpCbQUH40v+53v7s9P56CSNASirkFnmv89He0k/QOeoFnfB9TQf8nlIT3moAfbBUyC7oO9hc/lkYHWzGIWkDSVZWFg0NDRQVFSX8P3LnHA0NDWRlDfAMDOdg1b3ek+1u+tuI6n16qzda65YLNVprJDIzppeOZnrpaP7kmmkca+ng+d31rN9Rx8t7jnirOftBYPbYPH8Z/94PADuTWsrLTu+3JyPnKSPHe+WXR/6Z7i4vlda7p9NXQDpWA7VveduXfiq46/AlbSApLy+npqaG+vr6cxdOAFlZWZSXD/CP7s3H4J2X4f3/BLnFEdVZWRViVtlopimtlRTyc9L5wEXj+cBF4+PdFBmMlFSvl3G+Ey9jsJ5i0gaS9PR0Kiri95jZqGpphGf+2puEdfHHI/rI4eOtbH7nKJ9bqrSWyIgWg4xLkkyRHeGe/ZqXP731oYhvuD29tRbnYMWFGq0lIkOjQDLcvfs771kPS+6GsfMi/tiqqhAzSkcxvbT/pVNERCKhQDKcdXXAU5+HvAlw7cCLMoarO9HKprcbtdKviERFoIHEzJaZ2U4z22Nm9/dx/CEz2+K/dplZU9ixrrBjT4btrzCz35nZbjP7pZkNv9k70fK7f4G6bbD8m9749wit8dNaCiQiEg2BBRIzSwUeBpYDc4GVZjY3vIxz7vPOuQXOuQXA94DfhB0+1XPMOfeBsP3fBB5yzs0AjgLBj21LRMdqYP3fw8xlMPvW8/roqqoQ00pymVmm0VoiMnRB9kgWA3ucc/ucc+3A48AHByi/EnhsoArNm/BxPfArf9fPgA9Foa3Dz+ovectnL//WeY3KqD/Rxmv7G1kxf1zCz58RkeEhyEAyATgQtl3j73sPM5sMVADrwnZnmdlmM3vVzHqCRRHQ5JzrWY9hoDrv8j+/ebjMFYnYztWw4ym45otQOPm8PrpmWy3dDpYrrSUiURLkPJK+ft3tb2bMbcCvnHNdYfsmOecOmdlUYJ2ZVQHHI63TOfcI8AjAokWLgp+REyvtJ6Hyi1AyGy6757w/vnpriKnFucweq9FaIhIdQfZIaoCJYdvlwKF+yt5Gr7SWc+6Q/3MfsAG4GDgCFJhZTwAcqM6R6YVvw7F3YcV3z3uV0IbmNjbubWD5/LFKa4lI1AQZSDYBM/xRVhl4weLJ3oXMbBZQCGwM21doZpn++2LgCmC781YuXA98xC/6CeC3AV5DYqmr9hZlXHA7TLnivD++ZtthujVaS0SiLLBA4t/HuAdYA1QDTzjntpnZ180sfBTWSuBxd/bytnOAzWb2Jl7geNA5t90/9iXgXjPbg3fP5MdBXUNCcQ6eutd7jvWNXx9UFau3hphSlMPccXlRbpyIJLNA19pyzlUClb32PdBr+2t9fO4VYH4/de7DGxGWXLb8At59BT7wvYgXZQzXeLKdV/Y2cNfVU5XWEpGo0sz24aBnUcaJ74MFg3sk79rttXR1O1YorSUiUaZAMhysfcB7DsF5LMrY26qqWiaOyeaC8UpriUh0KZAkundfhd//HC77DJRdMKgqmlraeWXPEW7RJEQRCYACSSI7vShjOVzzpUFX88z2w3QqrSUiAUnaB1sNC6/+AOq2w22/OK9FGXurrApRXpjN/An5UWyciIhHPZJE1XQANjwIs26B2SsGXc2xlg5eVlpLRAKkQJKoVvuprOXfHFI1a6sP09HlWD5PT0IUkWAokCSiHZWwc5V3X6Rg0pCqqqwKMaEgmwUTC6LUOBGRsymQJJr2k7D6i1AyxxupNQTHWzt4cXc9y+dpbS0RCY5utiea578Jxw7AHz0NqelDqurZ7X5aS6O1RCRA6pEkksPbYePDcPEdMPmyIVdXWVXLuPwsLlZaS0QCpECSKLq7YdW9kJkHNwxuUcZwJ1o7eGF3PcvmjSUlRWktEQmOUluJYst/wLsb4QPfh9yiIVf3XHUd7Z3dmoQoIoFTjyQRnGyAtV+BSZd5zxqJgsqqEGV5mVwyqTAq9YmI9EeBJBGsfQDaTnhPPRzkoozhmts62bCrnuXzximtJSKBUyCJt3degS3/7i/KODcqVa7b4aW19CREEYkFBZJ46mz3nnqYP3FIizL2VvlWiNLRmSyarLSWiARPgSSeXn0Y6qvhlm9DRm5UqjzZ1sn6nXUarSUiMaNAEi9H34EN34RZK2DW8qhVu35nHW1Ka4lIDCmQxINz3jIoZkNelLG3yqoQxaMyuXTKmKjWKyLSHwWSeNixCnY9Ddd+GQomRq3alvZO1u+oZ9m8MlKV1hKRGFEgibW2Zm+J+NILYMndUa16w856TnV0Ka0lIjGlme2x9vyDcLwGPvLjIS/K2NuqqhBFuRksVlpLRGJIPZJYqt0KG38Al3wcJi2JatWn2rtYv6OOm+eNJS1Vf60iEjv6xomVnkUZswvghr+JevXP76qjpb2LW+YprSUisaXUVqz8/udw4HfwwR9ATvRTT5VVtYzJzWDJVKW1RCS21COJhZNHvPW0Jl8BCz4W9epbO7p4rvowN19QprSWiMScvnViYe0D0N7sLcoYwCNvn99Vz8n2LpYrrSUicaBAErS3X/KeNXL5Z6F0diCnWF0VoiAnncumDf05JiIi50uBJEg9izIWTIKrvxjIKVo7uni2uo6b5paRrrSWiMSBbrYHaeP34chO+NgTkJETyCle2n2E5rZOTUIUkbjRr7BBOfo2PP8tmH0rzLw5sNNUVoXIz07niunFgZ1DRGQgCiRBcA4qvwiWEvVFGcO1dXaxdvthblRaS0TiSN8+QdjxFOxeA9f9JeSXB3aal/cc4URbJyuU1hKROFIgiba2E96ijGXz4H2fDvRUq96qZXRWmtJaIhJXgQYSM1tmZjvNbI+Z3d/H8YfMbIv/2mVmTb2O55nZQTP7fti+DX6dPZ8rDfIaztuGB+H4Qbj1IUgNbixDe2c3a7fXcuPcMjLS9PuAiMRPYN90ZpYKPAzcCNQAm8zsSefc9p4yzrnPh5X/LHBxr2r+Fni+j+pvd85tjn6rh6i2Cl79ISz8JExcHOipXt57hOOtSmuJSPxF9Kusmf3azFaY2fn86rsY2OOc2+ecawceBz44QPmVwGNh51wIlAHPnMc546e7G576PGQXwtKvBn66yrdCjM5M48oZSmuJSHxFGhh+CHwM2G1mD5pZJFO0JwAHwrZr/H3vYWaTgQpgnb+dAvwDcF8/df/UT2t9xSyANUcG442fQc0muOkbgSzKGK6jq5tnth/mhrllZKalBnouEZFziSiQOOeedc7dDlwCvA2sNbNXzOyPzKy/pzP19QXv+il7G/Ar51yXv/2nQKVz7kAfZW93zs0HrvJfd/Z5crO7zGyzmW2ur6/v57RR0lwPz34NJl8JF90W7LmAV/Y2cOxUB8vnjQ38XCIi5xJxqsrMioBPAv8L+D3wT3iBZW0/H6kBwh9IXg4c6qfsbYSltYDLgHvM7G3gO8DHzexBAOfcQf/nCeAXeCm093DOPeKcW+ScW1RSUnKuyxuatV+B9pNwazCLMva2uirEqMw0rp4Z8HWJiEQgopvtZvYbYDbwc+D9zrmQf+iXZtbfTe9NwAwzqwAO4gWL96yhbmazgEJgY88+v/fTc/yTwCLn3P1mlgYUOOeO+D2hW4FnI7mGwOx/Ed58DK76ApTMCvx0HV3drNlWy9I5pWSlK60lIvEX6ait7zvn1vV1wDm3qJ/9nWZ2D7AGSAV+4pzbZmZfBzY75570i64EHnfO9Zf2CpcJrPGDSCpeEPnXCK8h+jrbvaceFkyGq/4iJqd8dV8DR1s6tGS8iCSMSAPJHDN7wznXBGBmhcBK59wPBvqQc64SqOy174Fe2187Rx2PAo/6708CCyNsc/Be+Wc4sgs+9p+BLcrYW2VVLbkZqVw7S2ktEUkMkd4j+d89QQTAOXcU+N/BNGmYaNwPL3wb5nwAZt4Uk1N2+mmt6+eUKa0lIgkj0kCSEj7M1p9smBFMk4YB56DyPkhJg2UPxuy0r+1vpPFkO7dotJaIJJBIU1trgCfM7F/whvB+Gng6sFYluuonYc9auPn/QH6fU2MCsaoqRHZ6KtfOSqxVYUQkuUUaSL4E/AlwN978kGeAfwuqUQmt7QSsvh/K5sPiP4nZabu6nZfWml1KdobSWiKSOCIKJM65brzZ7T8MtjnDwPq/hxMh+OjPA12UsbfX9jdypLldT0IUkYQT6TySGcDfA3OBrJ79zrmpAbUrMYXehN/5izKW9znqOTCVVSGy0lO4brZGa4lIYon0ZvtP8XojncB1wP/Fm5yYPLq74al7IXsM3BD8oozhurodT2+r5bpZpeRkxK4XJCISiUgDSbZz7jnAnHPv+HM/rg+uWQnojUfh4GbvBnt2YUxPvfntRupPtCmtJSIJKdJfb1v9FXl3+7PVDwLJM3Souc5blHHKVXDhH8b89JVVITLTUrh+dvL8kYvI8BFpj+RzQA7wZ3gzy+8APhFUoxLOM38N7S2wIjaLMobr7nas3lrLtbNKyM1UWktEEs85v5n8yYd/6Jy7D2gG/ijwViWS/S/AW7+Eq++DkpkxP/3r7x6lTmktEUlg5+yR+M8IWZgwD5CKpc427wZ74RRvdd84qKwKkZGWwtI5ZXE5v4jIuUSaK/k98Fsz+0/gZM9O59xvAmlVonj5n6FhN9z+a0jPjvnpu7sdq6tquWZmCaOU1hKRBBXpt9MYoIGzR2o5YGQHkhOHYO6HYMYNcTn97w8cpfZ4K1+aH/xzTkREBivSme3JdV+kx60PQVdn3E5fWVVLRqrSWiKS2CKd2f5T+njeunPuj6PeokQTw2VQwnlprRBXzywmLys9Lm0QEYlEpN+ST4W9zwI+TP/PX5coeLOmiUPHWvnCTUpriUhiizS19evwbTN7jHg/K32Eq6wKkZ5q3DBXaS0RSWyRTkjsbQYwKZoNkTOcc1RW1XLl9GLys5XWEpHEFuk9khOcfY+kFu8ZJRKAt2qOcbDpFJ+7YUa8myIick6RprZGB90QOaOyKkRainHTXD1SV0QSX0SpLTP7sJnlh20XmNmHgmtW8nLOUbk1xBXTi8nPUVpLRBJfpPdIvuqcO9az4ZxrAmL7UI4ksfXgcQ40nmKF1tYSkWEi0kDSVzmt2RGAVVUhUlOMGzVaS0SGiUgDyWYz+66ZTTOzqWb2EPB6kA1LRs45Vm8Ncfm0IgpzM+LdHBGRiEQaSD4LtAO/BJ4ATgGfCapRyWrboeO809CitJaIDCuRjto6CdwfcFuS3uqtXlrrpgs0WktEho9IR22tNbOCsO1CM1sTXLOST88kxMumFjFGaS0RGUYiTW0V+yO1AHDOHSWZntkeA9WhE+w/cpLl89UbEZHhJdJA0m1mp5dEMbMp9LEasAze6q0hUgxuVlpLRIaZSIfw/hXwkpk9729fDdwVTJOSj3OOVVUhlkwtonhUZrybIyJyXiLqkTjnngYWATvxRm59AW/klkTBrsPN7Ks/yXKN1hKRYSjSRRv/F/DnQDmwBVgCbOTsR+/KIK2q8tJay5TWEpFhKNJ7JH8OXAq845y7DrgYqA+sVUlmdVXvP7SvAAANlUlEQVSIxRVjKBmttJaIDD+RBpJW51wrgJllOud2AHp0XxTsPnyC3XXN3KK0logMU5HebK/x55H8F7DWzI6iR+1GxaqqEKa0logMY5HebP+wc67JOfc14CvAj4FzLiNvZsvMbKeZ7TGz98yMN7OHzGyL/9plZk29jueZ2UEz+37YvoVmVuXX+c9mZpFcQ6JaXVXLpZPHUJqXFe+miIgMynk/atc597xz7knnXPtA5cwsFXgYWA7MBVaa2dxedX3eObfAObcA+B7wm17V/C3wfK99P8QbejzDfy0732tIFHvqmtl5+AS3aBKiiAxjg31meyQWA3ucc/v8oPM48MEByq8EHuvZMLOFQBnwTNi+cUCec26jc84B/5cIekaJanVVCEDDfkVkWAsykEwADoRt1/j73sPMJgMVwDp/OwX4B+C+PuqsiaTO4WBVVYhFkwspU1pLRIaxIANJX/cu+ltW5TbgV865Ln/7T4FK59yBXuUirtPM7jKzzWa2ub4+8UYq76tvZkftCY3WEpFhL8inHNYAE8O2y+l/pNdtnP18k8uAq8zsT4FRQIaZNQP/5Ndzzjqdc48AjwAsWrQo4dYFW721FkCLNIrIsBdkINkEzDCzCuAgXrD4WO9CZjYLKMSbKQ+Ac+72sOOfBBY55+73t0+Y2RLgd8DH8W7SDzur3gpxyaQCxuVnx7spIiJDElhqyznXCdwDrAGqgSecc9vM7Otm9oGwoiuBx/2b55G4G/g3YA+wF1gdxWbHxNtHTrI9dFxpLREZEYLskeCcqwQqe+17oNf2185Rx6PAo2Hbm4F50WpjPFRu1WgtERk5grzZLv2orAqxYGIBEwqU1hKR4U+BJMbebWhh68HjmoQoIiOGAkmMnU5rzVNaS0RGBgWSGFtdFeKi8nwmjsmJd1NERKJCgSSGDjS28GbNMd1kF5ERRYEkhlb7aa1blNYSkRFEgSSGKqtqmTchj0lFSmuJyMihQBIjB5tOseVAkyYhisiIo0ASIz1LxiutJSIjjQJJjFRWhZg7Lo8pxbnxboqISFQpkMTAoaZTvPFuEysuVG9EREYeBZIYeLpnyfh5ms0uIiOPAkkMVFaFmD12NFNLRsW7KSIiUadAErDaY61sfucoKzRaS0RGKAWSgD2tJeNFZIRTIAlYZVUts8pGM71UaS0RGZkUSAJUd7yVTe80ahKiiIxoCiQBenpbLc6hZ4+IyIimQBKgyqoQM0pHMaNsdLybIiISGAWSgNSfaOO1/Y26yS4iI54CSUCe3lZLt0PDfkVkxFMgCcjqqhDTSnKZWabRWiIysimQBOBIcxuv7mvglvnjMLN4N0dEJFAKJAF4Ztthuh0a9isiSUGBJACVVSEqinOZPVajtURk5FMgibLGk+1s3NfALfPHKq0lIklBgSTKntlWS1e3U1pLRJKGAkmUraoKMbkoh7nj8uLdFBGRmFAgiaKjJ9t5Za9Ga4lIclEgiaK12w97aa15SmuJSPJQIImiVVUhJo7JZt4EpbVEJHkokETJsZYOXt5zRGktEUk6CiRR8sz2WjqV1hKRJKRAEiWrt9YyoSCbC8vz490UEZGYUiCJgmOnOnhxd70mIYpIUlIgiYJntx+mo0uTEEUkOQUaSMxsmZntNLM9ZnZ/H8cfMrMt/muXmTX5+yeb2ev+/m1m9umwz2zw6+z5XGmQ1xCJ1VtDjM/PYsHEgng3RUQk5tKCqtjMUoGHgRuBGmCTmT3pnNveU8Y59/mw8p8FLvY3Q8Dlzrk2MxsFbPU/e8g/frtzbnNQbT8fx1s7eGHXEe68bLLSWiKSlILskSwG9jjn9jnn2oHHgQ8OUH4l8BiAc67dOdfm788MuJ1Dsq66jvaubqW1RCRpBfkFPQE4ELZd4+97DzObDFQA68L2TTSzt/w6vhnWGwH4qZ/W+orFuRuwqirE2LwsLlZaS0SSVJCBpK8veNdP2duAXznnuk4XdO6Ac+5CYDrwCTMr8w/d7pybD1zlv+7s8+Rmd5nZZjPbXF9fP+iLGMiJ1g6e31XP8vljSUlRWktEklOQgaQGmBi2XQ4c6qfsbfhprd78nsg2vKCBc+6g//ME8Au8FFpfn3vEObfIObeopKRkUBdwLut21NHe2c0KpbVEJIkFGUg2ATPMrMLMMvCCxZO9C5nZLKAQ2Bi2r9zMsv33hcAVwE4zSzOzYn9/OnArsDXAaxhQZVWIsrxMLplUGK8miIjEXWCjtpxznWZ2D7AGSAV+4pzbZmZfBzY753qCykrgcedceNprDvAPZubwUmTfcc5VmVkusMYPIqnAs8C/BnUNAznZ1smGnfWsXDxJaS0RSWqBBRIA51wlUNlr3wO9tr/Wx+fWAhf2sf8ksDC6rRycdTvqaOvUaC0RkYQdVpvoKqtClIzOZOFkpbVEJLkpkAxCS3sn63fWsXzeWFKV1hKRJKdAMgjrd9TT2tHNci0ZLyKiQDIYlVtDFI/KZHHFmHg3RUQk7hRIztOp9i7WVdexbF6Z0loiIiiQnLcNO+s41dGlJyGKiPgUSM5T5dZainIzlNYSEfEpkJyH1o4unqs+zE0XjCUtVX90IiKgQHJent9VT0t7l9bWEhEJo0ByHiqrQhTmpLNkqtJaIiI9FEgi5KW16rhZaS0RkbPoGzFCL+4+QnNbp9bWEhHpRYEkQpVVIQpy0rlsWlG8myIiklAUSCLQ1tnFs9sPc9PcMtKV1hIROYu+FSPw0u4jnFBaS0SkTwokEVhVFSIvK43LpxXHuykiIglHgeQc2ju7Wbvdm4SYkaY/LhGR3vTNeA4v7znCidZObpk/Nt5NERFJSAok51BZFWJ0VhpXTi+Jd1NERBKSAskAOrq6eWb7YW6cW6a0lohIP/TtOICX9xzh2KkOLRkvIjIABZIBrK6qZVRmGlfN1GgtEZH+KJAMYEpxLndeNpnMtNR4N0VEJGGlxbsBiezua6fFuwkiIglPPRIRERkSBRIRERkSBRIRERkSBRIRERkSBRIRERkSBRIRERkSBRIRERkSBRIRERkSc87Fuw2BM7N64J1BfrwYOBLF5gwHuubkoGse+YZ6vZOdc+dc+jwpAslQmNlm59yieLcjlnTNyUHXPPLF6nqV2hIRkSFRIBERkSFRIDm3R+LdgDjQNScHXfPIF5Pr1T0SEREZEvVIRERkSBRIBmBms81so5m1mdlfxLs9QTOzZWa208z2mNn98W5PLJjZT8yszsy2xrstsWBmE81svZlVm9k2M/vzeLcpaGaWZWavmdmb/jX/TbzbFCtmlmpmvzezp4I8jwLJwBqBPwO+E++GBM3MUoGHgeXAXGClmc2Nb6ti4lFgWbwbEUOdwBecc3OAJcBnkuDvuQ243jl3EbAAWGZmS+Lcplj5c6A66JMokAzAOVfnnNsEdMS7LTGwGNjjnNvnnGsHHgc+GOc2Bc459wLeLwxJwTkXcs694b8/gfclMyG+rQqW8zT7m+n+a8TfHDazcmAF8G9Bn0uBRHpMAA6Ebdcwwr9gkp2ZTQEuBn4X35YEz0/xbAHqgLXOuRF/zcA/Al8EuoM+kQKJ9LA+9o3439qSlZmNAn4NfM45dzze7Qmac67LObcAKAcWm9m8eLcpSGZ2K1DnnHs9FudTIOnFzD5jZlv81/h4tyeGaoCJYdvlwKE4tUUCZGbpeEHkP5xzv4l3e2LJOdcEbGDk3xe7AviAmb2Nl6a+3sz+PaiTKZD04px72Dm3wH8l0xfpJmCGmVWYWQZwG/BknNskUWZmBvwYqHbOfTfe7YkFMysxswL/fTZwA7Ajvq0KlnPuy865cufcFLz/y+ucc3cEdT4FkgGY2VgzqwHuBf7azGrMLC/e7QqCc64TuAdYg3cD9gnn3Lb4tip4ZvYYsBGY5f/9firebQrYFcCdeL+h9vS8b4l3owI2DlhvZm/h/cK01jkX6HDYZKOZ7SIiMiTqkYiIyJAokIiIyJAokIiIyJAokIiIyJAokIiIyJAokIhEiZk1n7vUgJ//lZlNHeD4rcm0cq0MHwokIgnAzC4AUp1z+wYotgpvtnJOjJolEhEFEpEoM8+3zWyrmVWZ2Uf9/Slm9gP/mRhPmVmlmX3E/9jtwG/D6lhmZm/4z9B4DrxVbPGW97g1xpckMqC0eDdAZAT6H3jPvbgIKAY2mdkLeLPKpwDzgVK8FQR+4n/mCuAx8Jb0AP4VuNo5t9/MxoTVvRm4Cngi+MsQiYx6JCLRdyXwmL/i7GHgeeBSf/9/Oue6nXO1wPqwz4wD6v33S4AXnHP7AZxz4c9LqQOSaTFRGQYUSESir68l+QfaD3AKyAor19/aRVl+WZGEoUAiEn0vAB/1H6ZUAlwNvAa8BPxP/15JGXBt2Geqgen++43ANWZWAdArtTUTSIrny8vwoXskItH3/4DLgDfxehZfdM7VmtmvgaV4gWAX3pMJj/mfWYUXWJ51ztWb2V3Ab8wsBS+ddaNf7jrgy7G6EJFIaPVfkRgys1HOuWYzK8LrpVzhB5lsvHsmVzjnuvr5bBnwC+fc0hg2WeScFEhEYsjMNgAFQAbwLefco2HHbsZ74NS7/Xz2UqDDObclBk0ViZgCiYiIDIlutouIyJAokIiIyJAokIiIyJAokIiIyJAokIiIyJAokIiIyJD8f9AnouCSiDiKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_axis=np.log10(C_s)\n",
    "\n",
    "plt.plot(x_axis,np.array(accuracy_s[:,0]),label='l1')\n",
    "plt.plot(x_axis,np.array(accuracy_s[:,1]),label='l2')\n",
    "plt.legend()\n",
    "plt.xlabel('log(c)')\n",
    "plt.ylabel('accuracy')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "l2正则，C在等于0的时候准确率最高,随着C不断增大,准确率缓慢下降\n",
    "\n",
    "l1正则，c在等于1的时候准确率最高，随着C不断增大,准确率缓慢降低\n",
    "\n",
    "### 8. 根据6中得到的训练数据，训练RBF核SVM，并对超参数（C和gamma）进行超参数调优，并分析参数C和gamma对模型复杂度的影响，以及对训练误差和交叉验证得到的测试误差的影响。（20分）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [],
   "source": [
    "#超参数调优，正则参数C,RBF核函数的宽度\n",
    "from sklearn.svm import SVC\n",
    "def fit_grid_point_RBF(C,gamma,X_train,y_train,X_val,y_val):\n",
    "    SVC3=SVC(C=C,kernel='rbf',gamma=gamma)\n",
    "    SVC3=SVC3.fit(X_train,y_train)\n",
    "    accuracy_train=SVC3.score(X_train,y_train)\n",
    "    accuracy=SVC3.score(X_val,y_val)\n",
    "    print(\"C={} and gamma={}:,accuracy_train={}，accuracy_val={}\" .format(C,gamma,accuracy_train,accuracy))\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C=0.1 and gamma=0.1:,accuracy_train=0.709125，accuracy_val=0.6995\n",
      "C=0.1 and gamma=1.0:,accuracy_train=0.743，accuracy_val=0.7285\n",
      "C=0.1 and gamma=10.0:,accuracy_train=0.551375，accuracy_val=0.518\n",
      "C=1.0 and gamma=0.1:,accuracy_train=0.753375，accuracy_val=0.7365\n",
      "C=1.0 and gamma=1.0:,accuracy_train=0.833，accuracy_val=0.771\n",
      "C=1.0 and gamma=10.0:,accuracy_train=0.97925，accuracy_val=0.6425\n",
      "C=10.0 and gamma=0.1:,accuracy_train=0.79575，accuracy_val=0.7645\n",
      "C=10.0 and gamma=1.0:,accuracy_train=0.95575，accuracy_val=0.7735\n",
      "C=10.0 and gamma=10.0:,accuracy_train=0.99925，accuracy_val=0.6465\n",
      "C=100.0 and gamma=0.1:,accuracy_train=0.880375，accuracy_val=0.7725\n",
      "C=100.0 and gamma=1.0:,accuracy_train=0.994125，accuracy_val=0.7465\n",
      "C=100.0 and gamma=10.0:,accuracy_train=1.0，accuracy_val=0.647\n",
      "C=1000.0 and gamma=0.1:,accuracy_train=0.952875，accuracy_val=0.7505\n",
      "C=1000.0 and gamma=1.0:,accuracy_train=0.9995，accuracy_val=0.7425\n",
      "C=1000.0 and gamma=10.0:,accuracy_train=1.0，accuracy_val=0.647\n"
     ]
    }
   ],
   "source": [
    "accuracy_s=np.matrix(np.zeros(shape=(5,3)),float)\n",
    "\n",
    "#需要调整的参数\n",
    "gamma_s=np.logspace(-1,1,3)\n",
    "\n",
    "C_s=np.logspace(-1,3,5)#吧10的-1次方到10的3次方分成5份\n",
    "\n",
    "\n",
    "for i, oneC in enumerate(C_s):\n",
    "    for j,gamma in enumerate(gamma_s):\n",
    "        tmp=fit_grid_point_RBF(oneC,gamma,X_train_part,y_train_part,X_val,y_val)\n",
    "        accuracy_s[i,j]=tmp\n",
    "        \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/liu/anaconda3/lib/python3.7/site-packages/matplotlib/font_manager.py:1331: UserWarning: findfont: Font family ['sans-serif'] not found. Falling back to DejaVu Sans\n",
      "  (prop.get_family(), self.defaultFamily[fontext]))\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VNXd+PHPN3vYspCELewJq2xJWJTFoKXQVnCjVdvHQututZVaSn3c5WnV2p9UH32se6lPLVbsg1itSoWAOyRssu9L2BJCErInM3N+f9ybEELCTEImd5J837zmlbn3npn7zQ1zv3POufccMcaglFJKnU+Q0wEopZQKfJoslFJKeaXJQimllFeaLJRSSnmlyUIppZRXmiyUUkp5pclCKaWUV5oslFJKeaXJQimllFchTgfQXOLi4ky/fv2cDkMppVqVrKysk8aYeG/l2kyy6NevH5mZmU6HoZRSrYqIHPSlnDZDKaWU8kqThVJKKa80WSillPJKk4VSSimvNFkopZTySpOFUkoprzRZKKWU8qrN3GehVLMxBjwucFeCqwLcVeC2f7oqrPXVj3q31y1bBbEDYOBU6JTg9G+nVJNoslDOMKbWCbf65NvQCbk5tlfUKXu+k30l4Ke56buPgIGXWY8+F0NIuH/2E6CMMZRUujlVXEleSQWnSirJK6nkVEkl+fbz/JJKojqEktInhpQ+MQzu3pngIHE69HZPk4VqnILDsPFNKDpW6yTr7SRcZ7urAjxVzR9bcLh18g0OtZ4Hh9rLYWceIWEQ3qnO9ury9vbgsAvcXqdsUAgc3wx7V8LeVfDl/8Dnz0BIJPSbdCZ5xA8GaV0nRY/HUFhWVXPCP1VSYT0vruRUafW6SvKK7eellVS6PPW+V1hIEF07hhHTIYxN2QX8Y/0RADqGBTO6T3RN8hjTJ5roDmEt+WsqQIzx0zeoFpaWlmZ0uA8/MQYOfQlf/wm2/xMw0KGrDyfU6hN0C2wPCmk9J9qKIjjwuZ08PoG8Pdb6Lr2spqqBl8GAqdAhtsVDq3R5KCg9823fOvFX1Jzozzrxl1SSX1qJp4FTSKfwEGI7hhHbMYyu9s/YTmHEdrDXdQojtmN4zbYOYcGI/Tc0xnD4VBlZh06x/mAB6w/ls+N4EW57ZwPjO1rJo28MqX1jSIrvRJDWPppERLKMMWley2myUA1yVcCWd+CrF6xvxhFRkDIHxt0C0X2cjq7tyD8I+1ZZyWNfBpQXAgI9x5ypdfQeZyXMRiqtdJFXbJ3Ua77x1zT9VHCqpMr+aa0rKnfV+z4iENMhjJgOoXTtGF5z4q+uCVgn/urEEE5Mx1DCQ4Iv7LjUUVLhYlN2ARsOFbD+YD7rD+WTX2rVUDtHhDC6t1X7SO0bw+g+0XSJaPzxao80WaimKzoO616FrNehJBfih8D422DkdRDW0eno2ja3C45uOFPryM4E44awTpj+kynvnU5ut4nkhvYkr6TqrDb/uo+8kgrKq+pv8gkNFmLPOtGf+YZf+1G9LrpDWMD1Gxhj2H+yhPWHrJrH+oP57DxRhDFWcktO6ERq3xjG2M1XA+M71tRc1BmaLFTjZWfB1y/A1v8DjxsGTYfxt8OA9NbTxNOKuNwe8kur6pzkK85qAqooOkWfwiyGlWcyzr2RPpIDwCFPPJ96RrLGM4IvPcNxhXVp4ERvJYGY2us6hdE5PKRNnjiLyqvYdLiQ9YfyyTqYz4ZD+Zy2a0tRkaGkVPd99I1hVO9oOoVrt60mC+UbdxVse9fqj8heB2GdYcx/WE1NXQc6HV2r5vEYth07zZrduRw+VXp2c1BJJYVlVTT08YuKDD33JN8hlH5BJxhUso7EvC+JyfmK4KoSjAQjiWl2k9XlVvNVsJ4Ewfob7DtZzPqDBWTZTVe7c4oBCBIY3L1LTQJJ7RtD364d2mQSPR9NFur8Sk5C5uuQ+ap1ZVPsAKsWMeoGiOjidHStVmFpFWt255KxM5fVu3I5WVwBQFyn6m/4tdr8O9Zq6+9gd/7aTUOhwT7cL+uushL83pWw5xOr+Qpj9S31v9RKHkmXa/9SHYVlVWw8XFBT89h4qICiCqv2EdsxjJQ+0Yyxk8fIxCg6hLXtxKvJQtXv2Gb4+kX45m3rstaBl1lJImkaBOkN/Y3l8Ri2HC1k9c5cMnblsuFQPh5j1QwmJ8eRPjiBKYPiSOgc4f9gSk9ZHeR7P7Eu0T1tXXpK16QztY5+k6xLh1UNt8ewJ6e4puax/lA++3JLAAgOEob26FxT80jpE0NiTGSbqn1oslBnuF2w8wOrqeng5xDawapBjLsVEoY4HV2rk19SyZrduazemcua3bmcLK4EYGRiFOmD4rl0cDyjEqMJ8aV24C/GwMldZ2odBz4DVxkEhULv8dYlukmXQ/dR+iWhHvkllWw4nF9z2e7GwwWUVroBq5aY0ifaSh59YxjRK4qI0Oa98qslabJQUJYP6/8Ca1+GwsMQ1cfqi0i5ESJjnI6u1fB4DN8cKSRjZy4Zu3LYdLgAj4HoDqFMSY4nfXA8UwbFE9cpgO/GdlXAoa/OXGV1/BtrfWTsmXs7Bl4GXXo6G2eAcrk97DxRxPpDBWw4mE/WoXwO5pUC1pVlw3p0IcWueaT0jaFnVESrqX1osmjPcnZYtYjNb0FVKfSbbF36Ovi7ENR6vwG1pFMllazZlUvGzhzW7D7JqZJKRGBkryguHZxAul17CLTLSX1WnGM1We35xEogJdZVVsQPtfs6LoM+l0BYB0fDDGQniyusez7sK682ZxfUXKrcvUsEKX2j7TvOY7ioV5dmv++kuWiyaG88Htj9sZUk9q2y7nYe+X2rP6L7CKejC3huj2FzdoFde8hlc3YBxlgdnlPsvofJyXF0DeTaQ1MZAye22rWOlXDwC6s/Kzgc+l58pr+j23C9hPo8qtwedhwrIuvgqZp7P7LzywAICw7iol5dzrrrvFuXFujH8oEmi/ai/DRs/KvVaZ2/Hzr3hLE3QepPoGNXp6MLaHnFFTVXLq3ZlUt+aRUiMCoxmvTB8aQPTmBEr6jWW3toqspSOPSF1Um+5xPI3W6t79TNGoYk6XLr3hsdQdernNPldqe5ddf55iOFNWNj9YqOZEyty3aH9uhCWEjL9x9psmjr8vbC2pdgw1+hsggSx8GE22HorCYNC9EeuD2GjYcLWL0zh4xduXxzpBBjoGvHMC61O6anJMcT01EHqTvL6aNW4ti70qq1luZZ62tG0L0c+kxodyPoNkWly8PWo4U1NY8NB/M5WlgOQHhIECMTo2pqHyl9Yojv7P9jGhDJQkRmAM8AwcArxpgn6mxfBEy1FzsACcaYaHubG7B74ThkjJl1vn21i2RhjPWB/fpFq8kpKAQuusbqj+iV6nR0ASm3qMLqe9iVy6e7cykorSJIYHTvaNLtvoeLekbpIHS+8njg+Cb7KquVcPgra+6P0A7Qd6JV6xh4GcQN0iYrHx0rLKu56mr9oXy2HCmkym2dl3vHRp512e6Q7p2b/So7x5OFiAQDu4BpQDawDrjBGLOtgfJ3A2OMMT+1l4uNMT5fEN6mk0VlCWxaYiWJkzuhYzyk3QRpP4HO3Z2OLqC43B42Hi6ouXJpy5HTAMR1CmPKIKtpaUpynA5x3Vy8jqBrN1k5MIJua1Ve5bZqHwfPdJ7nFFk3d0aGBjMyMaomeaT0jSH2AmvCgZAsLgYeMcZMt5fvAzDGPN5A+S+Ah40xK+xlTRb5B2Hdy9blr+WF0GM0TLgDhl+tVf5acorKa26K+2z3SQrLrNpDSp+Ymr6HYT26aO2hJVSPoLvnE9i/+twRdJMuh8Sx2lTaCMYYjhSU1fR7rD+Uz7ajp3HZw7X3j+vI1MEJPDRzWJPePxCSxWxghjHmZnv5RmC8Meauesr2Bb4CEo0xbnudC9gIuIAnjDHL6nndrcCtAH369Ek9ePCgX36XFmWMdePcVy9YN9IhMGyWdVVT7/FatceqPWw4XEDGzhwyduay9ahVe4jvHM6lg6z7HiYnxRPVQU9IjmpwBN3O0H/ymXs7Ygfo/+tGKqt0882Rwpq7zqMjQ3nq+6Oa9F6BkCy+D0yvkyzGGWPurqfsAqxEcXetdT2NMUdFZACwErjcGLO3of21+ppFVbk1BMfXL8KJb6yb5lLnwtibISrR6egcd+J0de0hh093n6So3EVwkJDaJ4ZLB1sJYliPLq3mRqh2qawADnxq39vxCRQcstZH9z1T6+g/xRrbSrUYX5OFP0fIygZ611pOBI42UPZ64Ge1Vxhjjto/94lIBjAGaDBZtFqnj56ZO6I0DxKGwcxnYeQPIDTS6egcU+X2sP5gPhm7rEtbtx+zag/duoTznYu6kz44gYlJcURFau2h1YiMhqEzrYcxcGrfmXs7vnnb+gxIMCSmQcJQq9M8tIP1OQjraP30ZV1wmNZU/MCfyWIdkCwi/YEjWAnhh3ULichgIAb4sta6GKDUGFMhInHAROD3foy1ZRljjRb69Z+s4cE9buvu6gm3W3dbt9P/6McLy2ualj7fc5KiChchQUJq3xh+PWMw6YMSGNqjs9Ye2gIRawj8rgOtIWhqj6C7dyXs+ACqyqCqBEz9Ezg1/N5BTU80vq5rhyMh+C1ZGGNcInIX8BHWpbOvGWO2ishjQKYxZrld9AZgiTm7PWwo8KKIeIAgrD6Leq+ialVclbBtmdUfcXQ9hEdZfRFjb4bY/k5H1+Kq3B4yD+STsSuH1Ttz2XG8CLCGSvjeyB6kD47nkqQ4nR6zPQgOhb6XWI/LHjiz3hhwV1pXBFaVnUkgVWVNWFcKxcftdaXWclUpuMqbEG/YmQQS1qFOQmmGdSERAfelUW/KawnFOWfmjig+AV2TrXsjRt3Q7oaLPlpQxmp7zKXP9+RRbNcexvaLrel7GNxNaw+qBXk81oi8tRNIVem5SaWhdZWlZ5JRTZk6SctT/9zmDZMzNZmzkkp1TadOrSe2v/WlswkCoc9CHd1oNTVtecf6dpQ0zWpqGnBZuxkWutLlIfPAKbvvIYddJ6xZynpGRTBzVE/SB8czMSlOp7dUzgkKsk68/pxf3l11dg3nnKTSiHWlJ89NWt1HNjlZ+Eo/oc3N7YId71lXNR360vomkDrXmjsiLtnp6FrEkYKymr6HL/acpKTSTWiwVXuYnZpI+uAEkhM6ae1BtR/BoVYHf2S0f96/BVqINFk0l9JTkPVnWPeKNUNZTD+Y/jiM+VGbvxSwwuVm3f58K0HsymWPPcdxr+hIrhrTi/TBCVw8sKvWHpTylxb44qWf3gt1YqtVi9j8d6vds/+l8N0/wKDpbfqKicOnSsnYlcvqnTl8sTeP0ko3YcFBjOsfy/Vje5M+OJ6B8Vp7UKqt0GTRFB437PrQ6o/YvwZCImHUdTDuNujWtFvuW4MTp8v5vw1H+Mf67Jq+h96xkVybksilg+K5eGBXOmrtQak2ST/ZjVFeCBv+1xoaPP8AdEmEbz0CKXPa7EBp5VVuPtmew9tZh1mzKxePgdS+MTx4xTDSB8czIK6j1h6Uagc0Wfji5G6rqWnjm9ZlcH0ugW89CkOugOC2dwiNseacfjszm+WbjlJYVkWPqAjuSB/ItSmJDIhvX5f7KqU0WTTM47HnjngB9vzbugnnotnW/RE9RzsdnV/kFJWzbMMRlmZZzUzhIUFMH96d2amJTEyKa38zximlamiyqKuiGDb9zapJ5O22ppKcer81TWmneKeja3aVLg+fbD/B0qxsMnbl4vYYxvSJ5ndXj+B7I3vo2EtKKUCTxRmn9luXva5/AyoKrZnnrnkFhl0JIW1rohxjDFuPnmZpVjbvbjxCfmkV3bqEc+uUAVybkkhSgjYzKaXOpsni9FF4/1fW3BFBwTDsKmuCoUSvd7+3OieLK2qamXYcLyIsJIhvD+vG7NREJifHazOTUqpBmiwioq3mpim/grSfQpeeTkfUrCpdHlbtzOHtzGwydubg8hhG9Y5m4VUXMWtkT50gSCnlE00WYR3gZ2sDboTHC7Xt6GnezjrMuxuPcqqkkvjO4dw0qT+zUxNJ7tbZ6fCUUq2MJgtoM4niVEllTTPTtmOnCQsO4lvDEvh+am8mJ8cREtw+Bi9USjU/TRatXJXbQ8bOXJZmHWbljhyq3IYRvaJ47MrhzBzZk5iObatzXinlDE0WrdSO46dZmpnNso1HOFlcSVynMOZe0o9rUxMZ0r2L0+EppdoYTRatSH5JJcs3HWVpVjbfHCkkNFi4fIh1NdOlg+MJ1WYmpZSfaLIIcC63hzW7c3k7M5t/bz9BldswvGcXHp45jCtH9yJWm5mUUi1Ak0WA2nWiiKVZ2fzfhiPkFlUQ2zGMGyf0Y3ZqIsN6ajOTUqplabIIIIWlVSzfZF3NtCm7kJAgYeqQBL5vzy4XFqLNTEopZ2iycJjbY1izO5elWdms2HqCSreHId078+AVw7hydE/iOoU7HaJSSmmycMqenGK7mSmbE6criOkQyg/H92F2aiLDe3bROSKUUgFFk0ULKiyr4p+brauZNhwqIDhImDo4nkdnJXLZkG7azKSUCliaLPzM7TF8vuckS7Oy+WjrcSpcHgZ168T93x3KVWN6Ed9Zm5mUUoFPk4Wf7Mu1mpn+sf4Ix0+XExUZynVje/P91N5c1EubmZRSrYsmi2Z0uryK9zcfY2lWNlkH8wkSSB+cwEMzh3H50ATCQ4KdDlEppZpEk8UF8ngMX+zNY2nWYT7cepzyKg9JCZ247ztDuHpMLxK6RDgdolJKXTBNFk104GQJ76zP5p2sbI4WltMlIoTZqYnMTu3NqMQobWZSSrUpmiwaobjCxQd2M9PaA6cIEpicHM993x3KtGHdiAjVZialVNvk12QhIjOAZ4Bg4BVjzBN1ti8CptqLHYAEY0y0vW0O8IC97b+MMYv9GWtDPB7DV/vzWJqVzb++OU5ZlZsB8R359YzBXDMmke5R2syklGr7/JYsRCQYeB6YBmQD60RkuTFmW3UZY8y8WuXvBsbYz2OBh4E0wABZ9mvz/RVvXYfySq1mpvXZZOeX0Tk8hKvG9OL7aYmM6R2tzUxKqXbFnzWLccAeY8w+ABFZAlwJbGug/A1YCQJgOrDCGHPKfu0KYAbwNz/GS0mFiw++sZqZvt5/ChGYlBTH/OmDmT68uzYzKaXaLX8mi17A4VrL2cD4+gqKSF+gP7DyPK/t5YcY8XgMaw+cYmlWNh98c4zSSjf94zoyf/pgrh7Ti57Rkf7YrVJKtSr+TBb1tdOYBspeDyw1xrgb81oRuRW4FaBPnz5NiZHs/DKuf+krOoWHMGtUT2anJpLaN0abmZRSqhZ/JotsoHet5UTgaANlrwd+Vue16XVem1H3RcaYl4CXANLS0hpKROfVp2sHXv/JWCb070pkmDYzKaVUffw5ct06IFlE+otIGFZCWF63kIgMBmKAL2ut/gj4tojEiEgM8G17nV9MHZygiUIppc7DbzULY4xLRO7COskHA68ZY7aKyGNApjGmOnHcACwxxpharz0lIguxEg7AY9Wd3UoppVqe1DpHt2ppaWkmMzPT6TCUUqpVEZEsY0yat3I6gYJSSimvNFkopZTySpOFUkoprzRZKKWU8kqThVJKKa80WSillPJKk4VSSimvNFkopZTySpOFUkoprzRZKKWU8kqThVJKKa80WSillPJKk4VSSimvNFkopZTySpOFUkoprzRZKKWU8kqThVJKKa98ShYi8o6IfE9ENLkopVQ75OvJ/wXgh8BuEXlCRIb4MSallFIBxqdkYYz5tzHmR0AKcABYISJfiMhPRCTUnwEqpZRyns/NSiLSFZgL3AxsAJ7BSh4r/BKZUkqpgBHiSyER+QcwBHgDmGmMOWZvektEMv0VnFJKqcDgU7IAnjPGrKxvgzEmrRnjUUopFYB8bYYaKiLR1QsiEiMid/opJqWUUgHG12RxizGmoHrBGJMP3OKfkJRSSgUaX5NFkIhI9YKIBANh/glJKaVUoPG1z+Ij4O8i8ifAALcDH/otKqWUUgHF12SxALgNuAMQ4GPgFX8FpZRSKrD4lCyMMR6su7hf8G84SimlApGv91kkA48Dw4CI6vXGmAF+iksppVQA8bWD+3WsWoULmAr8BesGvfMSkRkislNE9ojIbxoo8wMR2SYiW0XkzVrr3SKy0X4s9zFOpZRSfuBrn0WkMeYTERFjzEHgERH5FHi4oRfYV0w9D0wDsoF1IrLcGLOtVplk4D5gojEmX0QSar1FmTFmdGN/IaWUUs3P12RRbg9PvltE7gKOAAleXjMO2GOM2QcgIkuAK4FttcrcAjxv37eBMSanMcErpZRqGb42Q90DdAB+DqQC/wHM8fKaXsDhWsvZ9rraBgGDRORzEflKRGbU2hYhIpn2+qt8jFMppZQfeK1Z2M1JPzDGzAeKgZ/4+N5SzzpTz/6TgXQgEfhURC6y7xbvY4w5KiIDgJUi8o0xZm+d2G4FbgXo06ePj2EppZRqLK/JwhjjFpFUu7+i7sn+fLKB3rWWE4Gj9ZT5yhhTBewXkZ1YyWOdMeaovf99IpIBjAHOShbGmJeAlwDS0tIaE5tSzc4YQ4W7gjJXGeWucsrcZTXPXR4XBoMx9sP+5zGemtdWLxsMGPDgOVO2zs+a11W/xv5oVr/e275q9lO93MC+PHiwwjGN2tdZ71FrXzWvqfsedfatGqd3597cOdq/w/X52mexAXhXRN4GSqpXGmP+cZ7XrAOSRaQ/Vh/H9Viz7dW2DLgB+LOIxGE1S+0TkRig1BhTYa+fCPzex1iVqleVp8o6iVefzF32ydxdTlmV/dNeV12mpnytbXXfo3pbuau8zZ/oBEFECCIIxFoOkqCa9cCZZXudiNSU8/U9pN6GCdWQcle53/fha7KIBfKAy2qtM0CDycIY47I7wz8CgoHXjDFbReQxINMYs9ze9m0R2Qa4gfnGmDwRuQR4UUQ8WP0qT9S+ikq1PR7jOevkW/tkXvcEXe4up9RVeva66vLuek709k+XcTUqJkGICIkgMiSSiGD7p70cExFDZEjkOduqt9feFhESQUhQyJmTon3yBM46SVavP+vEWutEK9aZlSDOvEftn9Xrq2Ov3tdZ+/Gyr3PK1fmp2i9pXMtS4EpLSzOZmToPU0sorCjkaPHRmm/kZe5zv23X/cZ9vm3VCaCxQoNCa07SHUI6nHViP+ekHRJxzgm/9vOadcFnnocHh+sJUrV5IpLly7xEvt7B/Trndk5jjPlpE2JTrdju/N3c+K8bKakqOW+5IAmq9xt5ZEgk8R3izzqpn3Wirz6ph1on7oa+sYcHhxMS5GvFWCl1oXz9tP2z1vMI4GrO7axWbdzpytPcs+oeIkMiWThxIR1DOzaYEEKDQvVbuVJtiK8DCb5Te1lE/gb82y8RqYDkMR7+89P/5GjxUV6b8RpjEsY4HZJSqgX5elNeXcmA3tjQjry4+UVWZ69mwbgFmiiUaod87bMo4uw+i+NYc1yodmBN9hpe2PgCswbO4rrB1zkdjlLKAb42Q3X2dyAqMB06fYjfrPkNQ2KH8OCEB7UfQql2yqdmKBG5WkSiai1H63hNbV9pVSm/WPULgoKCWDR1EREhEd5fpJRqk3zts3jYGFNYvWCP3dTg8OSq9TPG8MgXj7CvcB+/n/J7enWqOwakUqo98TVZ1FdOL3Jvw/6y7S/868C/+PmYn3NJz0ucDkcp5TBfk0WmiDwtIgNFZICILAKy/BmYcs7aY2tZlLWIaX2n8dOL9L5LpZTvyeJuoBJ4C/g7UAb8zF9BKeccLznO/DXz6dOlDwsnLtQObaUU4PvVUCVAvXNoq7ajwl3BvFXzqHBX8MzUZ+gY2tHpkJRSAcLXq6FWiEh0reUYEfnIf2EpJzz+9eNsydvCbyf9lv5R/Z0ORykVQHxthoqzr4ACwJ4z29sc3KoVWbprKe/sfodbRtzC5X0udzocpVSA8TVZeESkZngPEelHPaPQqtZpc+5mfvf175jYcyI/G61dUUqpc/l6+ev9wGcistpenoI997Vq3U6WnWRexjwSOiTw5JQnCQ4KdjokpVQA8rWD+0MRScNKEBuBd7GuiFKtmMvjYv7q+ZyuOM0b332DqPAo7y9SSrVLvg4keDPwCyARK1lMAL7k7GlWVSvzdNbTZJ7I5HeTfseQ2CFOh6OUCmC+9ln8AhgLHDTGTAXGALl+i0r53Qf7PuCNbW/wo6E/YubAmU6Ho5QKcL4mi3JjTDmAiIQbY3YAg/0XlvKnnad28vAXD5OSkMK9afc6HY5SqhXwtYM7277PYhmwQkTy0WlVW6XCikLmZcyjS1gX/l/6/yM0KNTpkJRSrYCvHdxX208fEZFVQBTwod+iUn7hMR7u+/Q+jpUc4/XprxMXGed0SEqpVqLRI8caY1Z7L6UC0QubXuDTI5/y4IQHGZ0w2ulwlFKtSFPn4FatTMbhDP606U9clXQV3x/0fafDUUq1Mpos2oGDpw9y36f3MazrMB6Y8ICOJKuUajRNFm1caVUpv1j5C0KDQvlj+h8JDw53OiSlVCuks921YcYYHvz8Qfaf3s+L016kR6ceToeklGqltGbRhi3eupiPD37MPSn3MKHHBKfDUUq1Ypos2qivj33NovXW1Khzh891OhylVCunyaINOlZ8jPmr59O/S3+dGlUp1Sz8mixEZIaI7BSRPSJS77SsIvIDEdkmIltF5M1a6+eIyG77McefcbYlFe4K7sm4hypPFX+c+kedGlUp1Sz81sEtIsHA88A0IBtYJyLLjTHbapVJBu4DJhpj8kUkwV4fCzwMpGFNspRlvzbfX/G2BcYY/uur/2Jb3jaenfos/aL6OR2SUqqN8GfNYhywxxizzxhTCSwBrqxT5hbg+eokYIzJsddPB1YYY07Z21YAM/wYa5vw9q63WbZnGbeNvI2pfaY6HY5Sqg3xZ7LoBRyutZxtr6ttEDBIRD4Xka9EZEYjXouI3CoimSKSmZvbvkdM35izkcfXPs6kXpO4Y9QdToejlGpj/Jks6utVrTtvdwiQDKQDNwCv2KPb+vJajDEvGWPSjDFp8fHxFxhu63Wy7CT3ZtxL9w7deWLyEzo1qlKq2fkzWWQDvWvCOP+cAAAV1klEQVQtJ3LusObZwLvGmCpjzH5gJ1by8OW1CqjyVHFvxr2crjzNH6f+UadGVUr5hT+TxTogWUT6i0gYcD2wvE6ZZcBUABGJw2qW2gd8BHxbRGJEJAb4tr1O1fF05tOsz1nPI5c8wuBYnY9KKeUffrsayhjjEpG7sE7ywcBrxpitIvIYkGmMWc6ZpLANcAPzjTF5ACKyECvhADxmjDnlr1hbq/f2vsf/bv9f/mPof/C9Ad9zOhylVBsmxpzTFdAqpaWlmczMTKfDaDE7Tu3gxg9u5KK4i3jp2y/pjHdKqSYRkSxjTJq3cnoHdytUWFHIPavuoUt4F5669ClNFEopv9NRZ1sZt8fNgjULyCnN4c8z/qxToyqlWoTWLFqZ/9n0P3x+9HPuG38fI+NHOh2OUqqd0GTRinxy6BNe2vwS1yRfw+zk2U6Ho5RqRzRZtBL7C/dz/2f3c1HXi/jP8f+pI8kqpVqUJotWoKSqhHtW3UN4cDiLpi7SqVGVUi1OO7gDXPXUqAdOH+DlaS/TvWN3p0NSSrVDWrMIcK9teY0VB1fwy9RfMq7HOKfDUUq1U5osAtgXR7/g2Q3PMqPfDH487MdOh6OUasc0WQSoI8VHWLBmAQOiBvDoJY9qh7ZSylGaLAJQuauceavm4fa4+ePUP9IhtIPTISml2jnt4A4wxhgWfrWQ7ae289xlz9G3S1+nQ1JKKa1ZBJq3dr7F8r3LuWPUHVza+1Knw1FKKUCTRUDZmLORJ9c+yaWJl3L7qNudDkcppWposggQuaW5/DLjl/To1IPfTf4dQaJ/GqVU4NA+iwBQ5a7i3tX3UlxVzJ+m/YkuYV2cDkmpC1ZVVUV2djbl5eVOh6KAiIgIEhMTCQ1t2pQGmiwCwFOZT7EhZwO/n/J7BsUMcjocpZpFdnY2nTt3pl+/fnrpt8OMMeTl5ZGdnU3//v2b9B7a1uGw5XuX87cdf+PHw37Md/p/x+lwlGo25eXldO3aVRNFABARunbtekG1PE0WDtqet53HvnyMsd3HMi91ntPhKNXsNFEEjgv9W2iycEhBeQHzMuYRHR7NU1OeIiRIWwSVUueaMWMG0dHRXHHFFQ2Wqaio4LrrriMpKYnx48dz4MCBZo9Dk4UD3B43v17za3JKc1iUvoiukV2dDkkpFaDmz5/PG2+8cd4yr776KjExMezZs4d58+axYMGCZo9Dk4UDntv4HF8e+5L7x9/PiPgRToejVJu1cOFChgwZwrRp07jhhhv4wx/+wMsvv8zYsWMZNWoU1157LaWlpQDMnTuXO+64g6lTpzJgwABWr17NT3/6U4YOHcrcuXNr3rNTp04sWLCA1NRUvvWtb7F27VrS09MZMGAAy5cvB+DAgQNMnjyZlJQUUlJS+OKLL5r8O1x++eV07tz5vGXeffdd5syZA8Ds2bP55JNPMMY0eZ/10baPFvbvg//mlW9eYfag2Vw76Fqnw1GqRTz63la2HT3drO85rGcXHp45vMHtmZmZvPPOO2zYsAGXy0VKSgqpqalcc8013HLLLQA88MADvPrqq9x9990A5Ofns3LlSpYvX87MmTP5/PPPeeWVVxg7diwbN25k9OjRlJSUkJ6ezpNPPsnVV1/NAw88wIoVK9i2bRtz5sxh1qxZJCQksGLFCiIiIti9ezc33HADmZmZFBUVMXny5HrjffPNNxk2bFiTjsWRI0fo3bs3ACEhIURFRZGXl0dcXFyT3q8+mixa0L7Cfdz/2f2MiBvBfePuczocpdq0zz77jCuvvJLIyEgAZs6cCcCWLVt44IEHKCgooLi4mOnTp9e8ZubMmYgII0aMoFu3bowYYdX8hw8fzoEDBxg9ejRhYWHMmDEDgBEjRhAeHk5oaCgjRoyo6SuoqqrirrvuYuPGjQQHB7Nr1y4AOnfuzMaNG5v9d62vFtHcFxdosmghxZXF/GLlL4gIieDp9KcJCw5zOiSlWsz5agD+0lAzzNy5c1m2bBmjRo3iz3/+MxkZGTXbwsOtKYuDgoJqnlcvu1wuAEJDQ2tOxLXL1S6zaNEiunXrxqZNm/B4PERERAB4rVkUFRVx2223AfDYY48xa9Ysn37XxMREDh8+TGJiIi6Xi8LCQmJjY316ra80WbQAj/Fw/2f3c7joMC9/W6dGVaolTJo0idtuu4377rsPl8vF+++/zy233EJRURE9evSgqqqKv/71r/Tq1avZ911YWEhiYiJBQUEsXrwYt9sN+FazaErNY9asWSxevJiLL76YpUuXctlll2nNojV6bctrrDy8kvlp8xnbfazT4SjVLowdO5ZZs2YxatQo+vbtS1paGlFRUSxcuJDx48fTt29fRowYQVFRUbPv+8477+Taa6/l7bffZurUqXTs2LHJ7zV58mR27NhBcXExiYmJvPrqq0yfPp2HHnqItLQ0Zs2axU033cSNN95IUlISsbGxLFmypBl/G4s0d4+5U9LS0kxmZqbTYZzjiyNfcPu/b2dG/xk8OflJvUlJtRvbt29n6NChjsZQXFxMp06dKC0tZcqUKbz00kukpKQ4GpOT6vubiEiWMSbN22u1ZuFH2UXZzF8zn6SYJB65+BFNFEq1sFtvvZVt27ZRXl7OnDlz2nWiuFB+TRYiMgN4BggGXjHGPFFn+1zgKeCIveo5Y8wr9jY38I29/pAxxreengBR5ipjXsY8DIZn0p/RqVGVcsCbb77pdAhtht+ShYgEA88D04BsYJ2ILDfGbKtT9C1jzF31vEWZMWa0v+LzJ2MMC79cyM5TO3nu8ufo3aW30yEppdQF8ecd3OOAPcaYfcaYSmAJcKUf9xcw/rbjb7y37z3uHH0nUxKnOB2OUkpdMH8mi17A4VrL2fa6uq4Vkc0islREan8FjxCRTBH5SkSu8mOczSrrRBZPrXuK9MR0bh15q9PhKKVUs/BnsqivN7fupVfvAf2MMSOBfwOLa23rY/fQ/xD4o4gMPGcHIrfaCSUzNze3ueJuspzSHO7NuJdenXvp1KhKqTbFn2ezbKB2TSEROFq7gDEmzxhTYS++DKTW2nbU/rkPyADG1N2BMeYlY0yaMSYtPj6+eaNvpCp3Fb/M+CWlrlIWpS+ic9j5B/5SSqn67N+/n/Hjx5OcnMx1111HZWVlveUef/xxkpKSGDx4MB999JHf4/JnslgHJItIfxEJA64HltcuICI9ai3OArbb62NEJNx+HgdMBOp2jAeUJ9c9yabcTTw28TGSY5KdDkcp1UotWLCAefPmsXv3bmJiYnj11VfPKbNt2zaWLFnC1q1b+fDDD7nzzjtr7hL3F78lC2OMC7gL+AgrCfzdGLNVRB4TkerLYH8uIltFZBPwc2CuvX4okGmvXwU8Uc9VVAFj2Z5lvLXzLX4y/CfM6DfD6XCUUrbWNkS5MYaVK1cye/ZsAObMmcOyZcvOKffuu+9y/fXXEx4eTv/+/UlKSmLt2rUXeLTOz6/3WRhjPgA+qLPuoVrP7wPOGX7VGPMF0Cometiat5WFXy5kfPfx/Dzl506Ho1Rg+tdv4Pg33ss1RvcR8J0nGtzcGocoT0hIIDo6mpAQ69ScmJjIkSNHzil75MgRJkyYULPcULnmpHdwX4D88nzmrZpHbGQsv7/09zo1qlIBpDUOUV7fhTr1jfzQEkOS16VntyZyeVzMXzOfvLI8/vKdvxAb0bzDASvVppynBuAvrXGI8qFDh1JQUIDL5SIkJITs7Gx69ux5TtnqIcmrNVSuOem1nU307IZn+frY1zww4QGGx7X8WP1KqfObNGkS7733HuXl5RQXF/P+++8DnDNEuT8UFhbSo0cPgoKCeOONN84Zory+x7BhwxARpk6dytKlSwFYvHgxV1557r3Ms2bNYsmSJVRUVLB//352797NuHHj/PK7VNNk0QQfHfiI17e8zg8G/YCrk692OhylVD1qD1F+zTXXnDNE+bRp0xgyZIhf9n3nnXeyePFiJkyYwK5duxo1RPmTTz7J008/TVJSEnl5edx0000ALF++nIcesrp8hw8fzg9+8AOGDRvGjBkzeP755wkODvbL71JNhyhvpL0Fe7nh/RtIjknm9emv64x3SjVAhygPPDpEeQspqizinlX3EBkSydOX6tSoSgU6HaK8+Wiy8FH11KjZRdm8/O2X6daxm9MhKaW80CHKm4/2Wfjo5c0vs+rwKn419lekdfdaY1NKqTZFk4UPPs3+lOc3Ps/3BnyPHw75odPhKKVUi9Nk4cXh04dZ8OkCBsUM4uGLH9apUZVS7ZImi/Moc5VxT8Y9CMKiqYuIDIl0OiSllHKEJosGGGN45ItH2J2/myenPEnvzjo1qlLK/5577jmSkpIQEU6ePNlgucWLF5OcnExycjKLFy9usFxz0auhGvDX7X/lg/0fcPeYu5nUa5LT4Sil2omJEydyxRVXkJ6e3mCZU6dO8eijj5KZmYmIkJqayqxZs4iJifFbXFqzqMe64+v4Q+YfmNp7KjePuNnpcJRSTdTahigHGDNmDP369TtvmY8++ohp06YRGxtLTEwM06ZN48MPP2z08WkMrVnUcaLkBL9a/St6d+7Nbyf9VqdGVaoZPLn2SXac2tGs7zkkdggLxi1ocHtrHKJ82LBhPv3uR44coXfvM03jOkR5C6t0V/LLjF9S7irntemv6dSoSrVirXGIcl/pEOUOe2LtE2w+uZmn059mYPRAp8NRqs04Xw3AX1rjEOW+1iwSExPPijs7O/u8fRzNQdtYbP/Y/Q/e3vU2P73op0zrO83pcJRSF6g1DlHuq+nTp/Pxxx+Tn59Pfn4+H3/88Vk1JH/QZAFsObmF3371Wyb0mMDdY+52OhylVDNorUOUP/vssyQmJpKdnc3IkSO5+WbrIpvMzMya57GxsTz44IOMHTuWsWPH8tBDDxEb698J2Nr9EOV5ZXlc//71BBHEkiuWEBPhv0vPlGpPdIjywKNDlF+AYAlmSMwQ7hh9hyYKpdoYHaK8+bT7ZBEdEc1/X/7fToehlPIDHaK8+WifhVJKKa80WSil/Kat9Im2BRf6t9BkoZTyi4iICPLy8jRhBABjDHl5eTX3ezRFu++zUEr5R/Xln7m5uU6HorCSd2JiYpNfr8lCKeUXoaGh9O/f3+kwVDPRZiillFJeabJQSinllSYLpZRSXrWZ4T5EJBc4eAFvEQc0PIehczSuxtG4Gkfjapy2GFdfY0y8t0JtJllcKBHJ9GV8lJamcTWOxtU4GlfjtOe4tBlKKaWUV5oslFJKeaXJ4oyXnA6gARpX42hcjaNxNU67jUv7LJRSSnmlNQullFJetatkISJDRORLEakQkV+dp1x/EflaRHaLyFsiEubnuEREnhWRPSKyWUTqnaFFRDJEZKeIbLQfCf6My97nDHufe0TkN/VsD7eP0R77mPULgJjmikhureN0s79jsvf7mojkiMiWBrb79Hd2IK50ESmsdbweaqG4eovIKhHZLiJbReQX9ZRp8WPmY1wtfsxEJEJE1orIJjuuR+sp47/PozGm3TyABGAs8FvgV+cp93fgevv5n4A7/BzXd4F/AQJMAL5uoFwGkNaCxysY2AsMAMKATcCwOmXuBP5kP78eeCsAYpoLPOfA/68pQAqw5UL+zg7ElQ7804Hj1QNIsZ93BnbV87ds8WPmY1wtfszsY9DJfh4KfA1MqFPGb5/HdlWzMMbkGGPWAVUNlRERAS4DltqrFgNX+Tm0K4G/GMtXQLSI9PDzPn0xDthjjNlnjKkElmDFWtuVWMcIrGN2uX0MnYzJEcaYNcCp8xRx5O/sQ1yOMMYcM8ast58XAduBXnWKtfgx8zGuFmcfg2J7MdR+1O109tvnsV0lCx91BQqMMS57ORv//0fpBRyutXy+fb5uV3sf9PNJ2de4asrYx6wQ6xg6GRPAtXazxVIR6e3HeBqjMX/nlnax3bzxLxEZ3tI7t5tLxmB9W67N0WN2nrjAgWMmIsEishHIAVYYYxo8Xs39edRkca76TsD+vmTM133+yBgzAphsP270a1S+xdXSx8uX/b0H9DPGjAT+zZlvWk5z4v+WL9ZjDfkwCvhvYFlL7lxEOgHvAPcYY07X3VzPS1rkmHmJy5FjZoxxG2NGA4nAOBG5qE4Rvx2vNp8sRORntTqhevrwkpNYVd3quT4SgaP+jMt+/9rffuvdpzHmiP2zCHgTq0nGn7J9iKumjH3MovBvk4fXmIwxecaYCnvxZSDVj/E0hi/Hs8UZY05XN28YYz4AQkUkriX2LSKhWCfkvxpj/lFPEUeOmbe4nDxm9j4LsPowZ9TZ5LfPY5tPFsaY540xo+2H1/9kxuoZWgXMtlfNAd71Z1xY30p+bF/5MQEoNMYcq11eREKq/zPa/5GvAOq9uqUZrQOSxbo6LAyrw2x5nTLLsY4RWMdspX0MHYupTpv2LKw250CwHC9/ZyeISPfqJk0RGYd1Xshrgf0K8Cqw3RjzdAPFWvyY+RKXE8dMROJFJNp+Hgl8C9hRp5j/Po8t1ZMfCA+gO1bmPQ0U2M+72Ns+AHrazwcAa4E9wNtAuJ/jEuB5rKt8vqHWFU/ARvtnRyAL2AxsBZ4BglvgmH0X62qQvcD99rrHgFn28wj7GO2xj9mAAIjpcfsYbcJK/ENa6P/X34BjWBdQZAM3AbcDt3v7Ozsc1121jtdXwCUtFNckrCaSzcBG+/Fdp4+Zj3G1+DEDRgIb7Li2AA/Z61vk86h3cCullPKqzTdDKaWUunCaLJRSSnmlyUIppZRXmiyUUkp5pclCKaWUV5oslGoEESn2Xuq8r18qIgPs551E5EUR2WuPIrpGRMaLSJj9PMTb+ynVUjRZKNVC7PGDgo0x++xVr2DdXZtsjBmONVpunLEGSPwEuM6RQJWqhyYLpZrAvqP4KRHZIiLfiMh19vogEfkfu6bwTxH5QESqRwP4EfZoACIyEBgPPGCM8QAYayTd9+2yy+zySgUEreYq1TTXAKOBUUAcsE5E1gATgX7ACKz5U7YDr9mvmYh1NzXAcKy7890NvP8WrLlXlAoIWrNQqmkmAX8z1iigJ4DVWCf3ScDbxhiPMeY41nAj1XoAub68uZ1EKkWkczPHrVSTaLJQqmkamkvkfHOMlGGN3QPWuEKjROR8n8FwoLwJsSnV7DRZKNU0a4Dr7Mlo4rGmLl0LfIY18VKQiHTDmn6z2nYgCcAYsxfIBB6tNXppsohcaT/vCuQaYxqc1VGplqTJQqmm+T+s0T83ASuBX9vNTu9gjey6BXgRa4a1Qvs173N28rgZayTkPSLyDdbcG9XD6E/FGglZqYCgo84q1cxEpJMxptiuHawFJhpjjttzEKyylxvq2K5+j38A9xljdrZAyEp5pVdDKdX8/mlPUhMGLLRrHBhjykTkYax5kg819GJ7UqdlmihUINGahVJKKa+0z0IppZRXmiyUUkp5pclCKaWUV5oslFJKeaXJQimllFeaLJRSSnn1/wGZMcqSYrNwYgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_axis=np.log10(C_s)\n",
    "for j,gamma in enumerate(gamma_s):\n",
    "    plt.plot(x_axis,np.array(accuracy_s[:,j]),label='gamma='+str(np.log10(gamma)))\n",
    "plt.legend()\n",
    "plt.xlabel('log(C)')\n",
    "plt.ylabel('accuracy')\n",
    "plt.savefig('RBF_SVM_Otto.png')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最优结果，C=10.0 and gamma=1.0:accuracy=0.7735\n",
    "gamma隐含地决定了数据映射到新的特征空间后的分布，gamma越大，支持向量越少，gamma值越小，支持向量越多。支持向量的个数影响训练与预测的速度。\n",
    "C损失函数项的系数,可视为控制最小训练误差和模型复杂度的参数。C越大，模型复杂度越大。\n",
    "\n",
    "C和gamma越大，训练集的准确率越高，但校验集的数据在C=10.0和gamma=1.0准确率最高，之后随着C和gamma增大而降低。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
